LLMs Compared - Which is better

Shalom_Karr · October 5, 2025, 2:36pm

Is that a question or a statement?

Shalom_Karr · October 5, 2025, 6:07pm

thinking
That would make sense

Dev-in-the-BM_2.0 · October 5, 2025, 8:44pm

It sounds like Gemini 3 is coming, and it will be big:

Google is likely to release Gemini 3 on October 9 after extensive testing on AI Studio. Early tests show significant performance on coding tasks.

ys770 · October 5, 2025, 8:52pm

fun chol hamoeid trip

Shalom_Karr · October 5, 2025, 9:39pm

I didn’t see anything in beta yet

Dev-in-the-BM_2.0 · October 5, 2025, 9:40pm

I guess if Google didn’t reach out to you to let you know about it then it’s probably not true.

Shalom_Karr · October 5, 2025, 9:42pm

AI

GEMINI 3 DROPS OCTOBER 9th!

Google’s latest AI model is hitting AI Studio with major upgrades:

Massive coding performance boost - outperforms current Gemini 2.5 & Anthropic Sonnet 4.5
Better SVG generation capabilities
New “My Stuff” gallery for your AI creations
Tighter Google app integrations
Agent mode with browser control coming soon

Perfect timing for developers and tech creators ready to level up their AI workflows. Mark your calendars!

#Gemini3 #GoogleAI #AIStudio

Dev-in-the-BM_2.0 · October 5, 2025, 9:46pm

Anything about it being released on October 9th is just internet hock with no real source or proof, just speculation.

Maybe yeah, maybe no, we’ll wait and see.

Dev-in-the-BM_2.0 · October 20, 2025, 7:54pm

Grok 5 is coming!

Elon Musk says Grok 5 will be released before the end of 2025.

Biden2020Prez · October 21, 2025, 4:21am

I’ve always been intrigued by this very common viewpoint. Personally, I cannot think of any personal data or info I’d be uncomfortable with Xi Jinping accessing, but comfortable with Sam Altman accessing.

In other words, if it’s truly private, I will not send it to any hosted llm, and if it’s not, I don’t care if China sees my stupid code I am trying to debug,

Biden2020Prez · October 21, 2025, 4:24am

Btw Qwen and Qwen coder are both really good, and are open weight.

I hate to say it, but China is leading OpenAI and all US companies in the industry of open AI. (Meaning, open source AI. Sorry, I had to do it.

anonymousfliphones · October 21, 2025, 4:33am

are they free?

ars18 · October 21, 2025, 5:01am

To self host? There’s definitely great free options but you have to have the hardware.

I’ve been using z.ai glm 4.6 as an add on to Claude code when I hit limits.

Cheapest plan is $3 a month. Pretty decent model

Dev-in-the-BM_2.0 · October 21, 2025, 5:23am

If you want to go through the effort, it’s way cheaper to pay for the cloud compute to run a model than to pay for whatever these companies are chaging for these models.

ars18 · October 21, 2025, 5:28am

I dunno… They are pretty cheap

Dev-in-the-BM_2.0 · October 21, 2025, 5:48am

Info and rankings of top self-hostable models

(I threw in ChatPT-5 and Claude 4.5 so you can see how the other models compare to it)

Perplexity AI:

Here is a ranked, 1–10 scoring table across key coding metrics and aspects, including a “relative to flagship” score that situates each model against GPT‑5 and Claude 4.5 on SWE‑bench Verified and agentic coding tasks.[1][2]

Ranking table

Model Open weights SWE‑bench Verified (%) Terminal‑Bench HumanEval pass@1 LiveCodeBench Self‑hostability (1–10) Agentic coding (1–10) Relative to flagship (1–10)

GPT‑5 (high) No [3] 72.5 [1] n/a [1] n/a [1] n/a [1] 1 [3] 10 [1] 10 [1]

Claude Sonnet 4.5 No [2] 70.6 [1] 61.4 (OSWorld; agentic proxy) [2] n/a [2] n/a [2] 1 [2] 10 [2] 10 [1]

Qwen3‑Coder‑480B‑A35B‑Instruct Yes (Apache‑2.0 per LB) [1] 67.0 [1] n/a [1] n/a [4] n/a [4] 5 (very large MoE) [1] 9 (frontier‑level reports) [5] 9 [1]

DeepSeek‑V3.1 Yes [6] 66.0 [1] n/a [1] n/a [7] Top performer claims (V3 family) [7] 6 (open, heavy MoE) [6] 9 (strong repo‑level reports) [7] 9 [1]

Kimi K2 Yes [8] 65.4 [1] 25.0 [9] n/a [10] n/a [9] 6 (open, sizable) [8] 7 (agentic model family) [10] 9 [1]

GLM‑4.5 Yes [11] 64.2 [12] 37.5 [9] n/a [12] n/a [12] 6 (32B active MoE) [12] 9 (strong terminal automation) [9] 8 [1]

Llama 4 Maverick (17B‑128E) Yes (Llama license) [13] 21.04 [1] n/a [1] n/a [14] n/a [14] 9 (17B active, efficient) [14] 4 (generalist, smaller) [1] 3 [1]

Falcon 2‑11B Yes [15] n/a [15] n/a [15] 29.6 [16] n/a [15] 10 (lightweight dense) [16] 3 (older training mix) [15] 3 [16]

BLOOM‑176B Yes [17] n/a [17] n/a [17] Low vs modern code models [18] n/a [19] 3 (very large dense) [19] 2 (not code‑tuned) [19] 1 [19]

How to read scores

The 1–10 “relative to flagship” score is normalized primarily to SWE‑bench Verified, where GPT‑5 and Claude 4.5 sit at the top of current public leaderboards, so models at 65–69% land near 9 and 70%+ at 10.[2][1]

“Agentic coding” reflects repo/terminal automation indicators (e.g., Terminal‑Bench, OSWorld for agentic computer use), along with model‑family claims and evaluation write‑ups.[9][2]

“Self‑hostability” weighs open weights availability and effective active parameters; small dense or small‑active MoE models score higher for local deployment feasibility.[12][13]

Key takeaways

Among open‑weights, GLM‑4.5, DeepSeek‑V3.1, Kimi K2, and Qwen3‑Coder reach mid‑to‑high 60s on SWE‑bench Verified, placing them within one tier of GPT‑5/Claude 4.5 for repository‑level coding.[12][1]

GLM‑4.5 shows particularly strong terminal automation scores, and Kimi K2 appears in multiple comparisons as a competitive agentic coder for practical use.[9][1]

Llama 4 Maverick is an efficient 17B‑active MoE that’s easy to self‑host, but its repo‑level coding score is far below the top tier; Falcon 2 and BLOOM are best seen as lightweight or legacy baselines rather than primary coders.[16][1]

Sources

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

Dev-in-the-BM_2.0 · October 22, 2025, 3:27pm

Gemini 3 started rolling out today!

Perplexity AI

ars18 · November 3, 2025, 3:07pm

It’s a bit dumber then CC (plan plan plan) but context and limits are much bigger. Never hit a limit yet and I abused it.

Worth a try if you are dipping your toes in the water and don’t want to pay much.

Dev-in-the-BM_2.0 · November 10, 2025, 5:53pm

Perplexity Pro for $12

1 year subscription for Perplexity Pro for only $12.

I highly recommend it, I have Perplexity Pro, love it, and use it all the time.

I can’t guarantee that this site is legit, you’ll have to figure that out yourself, but it looks good on Trustpilot.

https://www.trustpilot.com/review/cheapgpt.store?languages=all

Flippy · November 11, 2025, 12:05am

You can get a free year of Perplexity Pro with a Venmo account.

You like it better than ChatGPT?

Model	Open weights	SWE‑bench Verified (%)	Terminal‑Bench	HumanEval pass@1	LiveCodeBench	Self‑hostability (1–10)	Agentic coding (1–10)	Relative to flagship (1–10)
GPT‑5 (high)	No [3]	72.5 [1]	n/a [1]	n/a [1]	n/a [1]	1 [3]	10 [1]	10 [1]
Claude Sonnet 4.5	No [2]	70.6 [1]	61.4 (OSWorld; agentic proxy) [2]	n/a [2]	n/a [2]	1 [2]	10 [2]	10 [1]
Qwen3‑Coder‑480B‑A35B‑Instruct	Yes (Apache‑2.0 per LB) [1]	67.0 [1]	n/a [1]	n/a [4]	n/a [4]	5 (very large MoE) [1]	9 (frontier‑level reports) [5]	9 [1]
DeepSeek‑V3.1	Yes [6]	66.0 [1]	n/a [1]	n/a [7]	Top performer claims (V3 family) [7]	6 (open, heavy MoE) [6]	9 (strong repo‑level reports) [7]	9 [1]
Kimi K2	Yes [8]	65.4 [1]	25.0 [9]	n/a [10]	n/a [9]	6 (open, sizable) [8]	7 (agentic model family) [10]	9 [1]
GLM‑4.5	Yes [11]	64.2 [12]	37.5 [9]	n/a [12]	n/a [12]	6 (32B active MoE) [12]	9 (strong terminal automation) [9]	8 [1]
Llama 4 Maverick (17B‑128E)	Yes (Llama license) [13]	21.04 [1]	n/a [1]	n/a [14]	n/a [14]	9 (17B active, efficient) [14]	4 (generalist, smaller) [1]	3 [1]
Falcon 2‑11B	Yes [15]	n/a [15]	n/a [15]	29.6 [16]	n/a [15]	10 (lightweight dense) [16]	3 (older training mix) [15]	3 [16]
BLOOM‑176B	Yes [17]	n/a [17]	n/a [17]	Low vs modern code models [18]	n/a [19]	3 (very large dense) [19]	2 (not code‑tuned) [19]	1 [19]