r/AIToolsPerformance

GPT-5.6 Sol price cut by 50% on OpenRouter - enough to switch back from Flash?

OpenRouter cut GPT-5.6 Sol pricing by 50%, the HN thread from Monday pulled 625 points and 449 comments, so yeah, people noticed. And the timing matters, because the budget side of the catalog got crowded this month: Gemini 3.7 Flash at $0.38/M input and $1.88/M output with 1048k context, DeepSeek V4 Pro 0813 at $0.66/$1.98 also 1048k, Qwen3.8 2.4T A95B at $2.00/$6.00 with the same 1048k. All per the OpenRouter model data pulled this week.

Half price sounds dramatic until you ask half of what. The cut puts Sol back in the conversation against Flash at $1.88/M output and V4 Pro at $1.98/M output, both with 1048k context. If the halved rate still sits comfortably above them, the long context crowd has little reason to move. If it lands at parity, different conversation.

The other half of the story is speed. The same week, Cerebras published their post on accelerating Sol Ultrafast, 712 points on HN, so Sol is being pushed on two fronts at once: fast on Cerebras silicon, cheap on OpenRouter. Reads like a volume play more than anything.

Anyone moving coding work back to Sol after this cut, or staying on Gemini 3.7 Flash / DeepSeek V4 Pro for the 1M context and calling it a day?

reddit.com
u/IulianHI — 19 hours ago

Qwen3.8 27B at $3.20/M output - worth it over Gemini 3.7 Flash at $1.88 with 1M ctx?

The 27B hit OpenRouter on the 14th and the local crowd went for it. The official repo on HuggingFace shows 415k downloads and about 10.6k likes, the unsloth GGUF is past 2.7M downloads, and the FP8 build adds another 495k. That's a lot of people grabbing weights to run themselves.

The OpenRouter listing is where it gets weird for me: $0.45/M input, $3.20/M output, 262k context. Per the same listing, Gemini 3.7 Flash went up the day before at $0.38/M in and $1.88/M out with 1048k context, and DeepSeek V4 Pro 0813 is $0.66/$1.98, also 1048k. So the small dense Qwen wants about 1.7x Gemini's output price for a quarter of the context window.

The family pricing doesn't explain it either. The 2.4T A95B flagship, same OpenRouter data, is $2.00/$6.00 with 1048k context. Its output token costs under 2x the 27B's, which is backwards from how these tiers usually stack.

Best guess from the download split: the 27B's real user runs quants locally and the API price is aimed at people who can't be bothered. Fair enough, but then the question is what the hosted one is actually for.

Anyone here paying the $3.20 for the API instead of running the GGUF, and if so, what broke locally?

reddit.com
u/IulianHI — 3 days ago

GLM-5.3 benchmark breakdown: where it actually beats Fable 5 / GPT-5.6 Sol, and where the "beats the frontier" headlines are wrong

Z.ai dropped GLM-5.3 on August 14. I went through the launch materials, the benchmark table, and the launch-day coverage. Posting the breakdown here because the interesting part isn't the headline score, it's how they got it.

  • Same base model as GLM-5.2. ~744B MoE, ~40B active per token, 1M context, 128K max output. Not one new parameter. Z.ai's own framing: "Scaling post-training is all we did for GLM-5.3."
  • Terminal-Bench 3.0: 4.6 → 28.3. DeepSWE v1.1: 46.2 → 66.9. SWE-Marathon v1.1: 19.4 → 42.5. AutomationBench: 26.2 → 48.2.
  • ~50% better on Z.ai's internal Code Bench, reportedly while burning fewer output tokens than 5.2.
  • CyberGym 84.5%, the top number in Z.ai's table, narrowly. ExploitBench more than doubled (24.4 → 54.4).
  • Weights are not out. ~2 weeks, gated on a safety review. Available today via GLM Coding Plan and ZCode; standalone API listed as "coming soon" in the docs.
  • Still text-only. No vision, despite that being the loudest community ask before launch.

1. The method is the story

Almost every point release bundles architecture changes, new pretraining data, and post-training tweaks together, so you can never tell which change bought which point. Z.ai explicitly froze the base model this time and only scaled post-training: more executable environments, more long-horizon task variety, more RL compute.

The training setup is the part I'd want a paper on. Per Z.ai, agents generated sandbox environments modeled on real software projects, then wrote tasks against those environments, and a separate judge agent verified each task was actually solvable before it was handed to the model. Some tasks were sized at multiple days of senior-engineer work. Two named pieces of infra: slime (training to inference handoff) and SAO (asynchronous RL).

That design choice explains the shape of the gains. The metrics that exploded are the agentic, terminal-native, multi-file, multi-step ones. Static knowledge benchmarks moved far less.

2. The numbers

All figures below are Z.ai-reported, from their own comparison table. No independent reproductions exist yet.

Benchmark GLM-5.2 GLM-5.3 Kimi K3 Claude Fable 5 GPT-5.6 Sol
Terminal-Bench 3.0 4.6 28.3 n/a 33.7 34.6
DeepSWE v1.1 46.2 66.9 67.5 69.7 n/a
FrontierSWE 67.5 78.1 n/a 88.2 n/a
SWE-Marathon v1.1 19.4 42.5 n/a n/a n/a
AutomationBench 26.2 48.2 n/a n/a n/a
Agents' Last Exam 23.8 28.5 27.6 n/a 28.6
HLE (with tools) 54.7 62.5 59.8 63.9 64.5
CyberGym 77.2 84.5 n/a 83.8 * 83.6
ExploitBench 24.4 54.4 n/a 78.0 76.5
ExploitGym (2h / 6h) 29 / 39 105 / 130 n/a n/a 216 / 293
GDPval-AA v2 n/a 1769 n/a 1743 1730

* Small inconsistency worth flagging: Z.ai's developer docs attribute the 83.8 CyberGym score to Mythos 5, while the launch table is reported elsewhere as Fable 5. Same underlying family, different label. If you're citing this number, cite it carefully.

3. Where it loses (because the headlines aren't saying this)

"Best open-weights model" and "best model" are two different claims, and only the first one holds up. In Z.ai's own table, Fable 5 and GPT-5.6 Sol still lead on Terminal-Bench 3.0, DeepSWE, FrontierSWE, SWE-Marathon and ExploitBench, several of those by a wide margin. ExploitBench isn't close: 54.4 vs 78.0. ExploitGym isn't close either: 105 tasks in a 2h budget vs 216.

Kimi K3 also still edges it on DeepSWE (67.5 vs 66.9), and Kimi is roughly 3x the parameter count, which cuts the other way and is arguably the more impressive framing for GLM.

Real wins: GDPval-AA v2 (1769, ahead of everything in the table, across 44 occupation types) and CyberGym, narrowly.

4. The cyber result is the actual headline

Z.ai says the security capability "grew faster than anticipated" during training, which reads a lot more like an internal flag that got surfaced than like marketing copy.

Supporting numbers: across 269 open-source projects, GLM models found 2,436 distinct vulnerabilities, 1,097 rated critical or high. 53 had public CVEs at launch; the rest are under embargo. Reported scope spans kernels, OSes, browsers and protocols, including at least one bug that had apparently been sitting there since 1981. Launch-day story that got the most traction: a security researcher flagged that it found a serious flaw in Cursor.

And this is why the weights are late. GLM-5.2 shipped MIT-licensed weights to HF essentially immediately. GLM-5.3 gets a staged release: selected security partners first, broader access after, weights in ~2 weeks pending safety evaluation. For a lab whose entire competitive identity is fast and permissive, voluntarily sitting on the weights is a genuinely unusual move and probably more newsworthy than any single benchmark row.

License isn't confirmed for 5.3 yet. Don't assume MIT just because 5.2 was.

5. Access and pricing

  • GLM Coding Plan (live now, rolled out to existing subscribers): Lite ~$18/mo, Pro ~$72/mo, Max ~$160/mo. Trackers disagree slightly on Pro/Max (some list $80/$168) and promos shift constantly, so check the subscribe page rather than trusting any secondhand table, mine included.
  • ZCode: live.
  • Standalone per-token API: Z.ai's docs still say coming soon, and no GLM-5.3 per-token rate is published. GLM-5.2's $1.40 / $4.40 per 1M (cached input ~$0.26) is the only reference point, not a guarantee.
  • Weights: ~2 weeks out, safety review pending.
  • Reasoning effort is exposed as low / high / max. Reportedly can't be turned off entirely.

6. If you're evaluating it

Headline scores won't tell you much here, especially since the biggest claimed gain is on a private benchmark nobody outside Z.ai can audit. Things actually worth measuring on your own repos:

  1. A repo-scale task requiring navigation across many files
  2. Whether tool calls stay correct over a long chain, not just the first 20
  3. Structured output against a strict JSON schema
  4. Human correction rate before delivery, the metric that actually predicts whether you'll keep using it
  5. Token burn per completed task, not per response. Z.ai's efficiency claim is the one I most want to see independently checked

Discussion

  • Does anyone have hands-on numbers on the token-efficiency claim? That's the one that changes cost math the most, and it's the least verifiable right now.
  • Is the two-week weight delay meaningful safety practice, or does it not matter much given that the weights ship regardless and capability keeps diffusing downward in model size?
  • The "they're just distilling" explanation gets weaker every release. Faster release cycles, RL environment quality, and a maturing data market all seem like better explanations. What's your read?
  • Anyone still running 5.2 locally who plans to stay on it?

Sources

Every benchmark number above is vendor-reported. Treat accordingly until third parties reproduce them.

u/IulianHI — 5 days ago

AMD acquired Taalas to etch LLMs into silicon, 17,000 tok/s but what's the catch

AMD bought Taalas last Thursday, the Toronto startup that hardwires model weights directly into silicon instead of running them on GPUs. It was the top AI story on HN all week, 941 points and over 700 comments.

The demo that put Taalas on the map was their HC1 chip running Llama 3.1 8B at 3/6-bit quantization, hitting 17,000 tokens per second. Per the coverage from February when they came out of stealth, they also claimed 10x lower ownership cost and 10x faster inference compared to GPU-based systems.

The catch, and it's a big one: each chip runs exactly one model. The weights are physically etched at manufacturing time. You can't swap to a new model without fabricating new silicon. So if you picked Llama 3.1 8B and a better 8B drops next month, your chip still runs the old one.

That's why this sits awkwardly next to hosted API pricing. It's not competing with OpenRouter-style flexibility. It's a fixed-function accelerator, closer to an ASIC for one specific model than a general inference backend. The speed is real but the tradeoff is steep.

Anyone here actually looked into hardwired inference for a production workload, or is this still firmly in the "cool demo, not deployable" zone for most teams?

reddit.com
u/IulianHI — 10 days ago

Why does NVIDIA Nemotron 3.5 Lightning free tier get 1M context when paid only has 262k

NVIDIA put Nemotron 3.5 Lightning on OpenRouter yesterday and the tiering is kind of backwards. The free version gives you 1M context. The paid version, $0.10/M input and $0.25/M output, caps at 262k context. Usually it's the other way around.

The paid pricing puts it right next to DeepSeek V4 Flash at $0.08/M input and $0.25/M output per the OpenRouter listing from the start of this month. Same output price, slightly more on input, but less than a quarter of the context window. DeepSeek V4 Flash gives you 1M context on the paid tier.

So the real question is what the free tier actually limits. OpenRouter listings don't spell out rate caps or quality differences, but 1M context for free on a Lightning-branded model, which is NVIDIA's speed-optimized line, is aggressive. Upstage Solar Pro 4 is cheaper on paper at $0.03/M input and $0.12/M output, but that's only 524k context and it's paid.

If you're picking a 1M context model for cheap agentic workloads right now, DeepSeek V4 Flash is the safer bet at $0.25/M output with paid reliability. The Nemotron free tier feels more like a sandbox. Anyone actually run Nemotron 3.5 Lightning free for real workloads, or is the rate limiting too tight to be useful beyond testing?

reddit.com
u/IulianHI — 8 days ago
▲ 5 r/AIToolsPerformance+5 crossposts

I did a comparison between opus 5, opus 4.8 and fable 5

On paper you might think, since fable 5 is a mythos class model, it should perform better overall. Thats far away from the truth, actually in a 1shot prompt test, opus 4.8 stood above fable 5

Few findings were

  1. Fable 5 costs 2x on both, per million input/ output tokens in comparison to claude
  2. In performance wise, on test like arc agi, humanites last test, browse comp, opus 5 does better than fable 5

Finally i gave both of these models, 1 shot prompts to create a video game, within 10 minutes and fable 5 actually lost

Check my full explanation here

Fable 5 vs Opus 5 vs Opus 4.8
https://youtu.be/0fEr4kTly44

PS: I am still finding my audience, so let me know if you are native english or hindi speakers

Would really appreciate honest feedback!

u/ExamImmediate8956 — 13 days ago
▲ 35 r/AIToolsPerformance+13 crossposts

I built SpecJudge — a CLI that reads your specs and tells you which AI model to use without overpaying

Picking an AI model to implement a project is a coin flip between two costs: too powerful and you overpay for capability you won't use; too weak and it just can't do the job.

SpecJudge reads your Spec-Driven Development artifacts and recommends the model that's right-sized for the work — not the cheapest, not the most powerful, the one that actually fits. A local model (via Ollama) judges how hard your project is, then it's crossed against a catalog of models and prices to produce a ranked podium.

It's local-first (your specs never leave your machine), MIT-licensed, and the whole model catalog is plain YAML you can edit with a PR — no Python needed to add a model or fix a price.

pip install specjudge
specjudge /path/to/your/project      # add --open for a visual matrix in your browser

Requires Python 3.11+ and Ollama with a local model.

It's an early release — I'd genuinely like to hear if the ratings match your experience, and PRs to the catalog are very welcome.

u/jokiruiz — 14 days ago

Local AI doesn’t replace Claude—but my 24 GB Mac mini became a much better complement than expected

I rely heavily on Claude, but I recently tested whether the M4 Pro Mac mini I already own could handle useful local models without becoming a dedicated AI appliance. I saw what Network Chuck did with the $50K Mac Ultra 4 node cluster and was hoping to not need the same.

It could. A GPT-OSS 20B MoE model ran at roughly 63.9 tok/s on 24 GB unified memory, while a smaller 9B dense model ran around 44.8 tok/s. The result is a useful reminder that model architecture matters: MoE models may have large total parameter counts while activating substantially fewer parameters for each generated token.

My takeaway is not “cancel your cloud AI subscription.” It is that local models can be a compelling companion for private experiments, quick generation, offline work, and workloads where you want direct control over the model runtime.

I captured the full test, including MLX vs. GGUF performance and the impact of my running containers: https://www.youtube.com/watch?v=9_-bT62YWAI

How are you dividing work between Claude and local models today?

u/silent_lurker_69 — 11 days ago
▲ 19 r/AIToolsPerformance+4 crossposts

Benchmark notes: Kimi K3 reaches 88/98; Qwen3.8-Max is stronger than its score looks; Gemini 3.6 Flash slips vs 3.5

I ran Kimi K3, Qwen3.8-Max and Gemini 3.6 Flash on the current 98-task MindTrial set with the same Python executor available to all models.

A few results stood out.

Kimi K3 was the biggest surprise: 88/98, up from 66/98 for Kimi K2.6, with hard errors dropping from 22 to 1. That ties Claude Opus 5 for the highest raw pass count among single models in this set. The downside is runtime: about 8h48m summed across the tasks.

Qwen3.8-Max finished at 74/98, but that number hides a much stronger solver. It went 39/39 on text and had 92.5% accuracy on completed tasks. Of its 18 hard errors, 15 were response-parsing failures; in a number of those cases the useful answer was there, but the outer response was malformed. So strict output discipline is currently a major weakness.

There was also an interesting API/agent interaction with both Kimi and Qwen. In the original runs using provider-enforced JSON object/schema output, neither model used the supplied Python tool across the benchmark. I reran using API response_format: text while keeping the same JSON schema in the prompt. Tool use came back immediately: 387 calls for Kimi and 423 for Qwen, with much better task performance. For Qwen, the constrained-output path also seemed to interfere with how reasoning translated into actions/final answers.

Gemini 3.6 Flash was less impressive: 74/98 versus 77/98 for Gemini 3.5 Flash. It was faster (~1h45m vs ~2h13m) and used fewer tokens overall, but made more Python calls (712 vs 597). In this workload the extra tool use often looked exploratory rather than efficient.

The strict scores are still the scores—I did not repair malformed model answers after the fact. I just found the differences in failure mode and tool behavior interesting.

Results/data: http://www.petmal.net/shared/mindtrial/results/2026-08-06/mindtrial-eval-all-models-03-2026_26.html

u/Correct_Tomato1871 — 12 days ago

What is the best AI image upscaler?

An upscaler shouldn't just make an image bigger. A lot of them technically increase the resolution but add strange textures or make faces look artificial.

What I usually look for is whether the details become clearer without changing the original image too much. Facy AI did a surprisingly good job with that for portraits and older photos because the results looked sharper while still feeling natural.

reddit.com
u/Icearp — 14 days ago