u/Jonathan_Rivera

If you are using something similar in your setup please refer to it as pre-deterministic routing or something similar. You get the idea.

Too much astro-turfing going on for both of those right now.

The new easy setup feature is presented automatically upon downloading the app, or you can access it within the Providers section in Settings.

GitHub merged 57 pull requests in NousResearch/hermes-agent on August 29, 2026 (UTC). I grouped them by area below and sorted each section newest to oldest by merge time.

Models, providers & routing

• #98034 fix(models): accept live Nous Portal recommendations in /model validation (salvage #71315)
• #98000 fix(custom): omit Ollama-only think=false on strict OpenAI-compat endpoints
• #97917 feat(providers): add Tencent TokenPlan + hy4-preview default (salvage #96939)
• #97916 feat(providers): add Nebius Token Factory provider (salvage #28253)
• #97915 feat(providers): add Ramp Router (router.com) provider plugin (salvage #93548)
• #97827 fix(opencode): delisted keyless free models no longer 401 the picker — catalog revalidates live (#95914, salvage #95943)
• #97746 refactor(vertex): fold _creds_cache_key into _sa_snapshot (post-#97701 simplify)
• #97708 fix(cache): preserve the configured 1h TTL on the OpenCode Go route (salvage of #97582)
• #97701 fix(vertex): pick up rotated service-account files — signature-keyed creds cache
• #97710 fix(prompt-caching): tool-using sessions no longer 400 behind LiteLLM Anthropic proxies (salvage of #89931)
• #97612 Add tencent/hy4-preview to model pickers

Prompts, skills & agent behavior

• #98030 fix(compression): arm the failure cooldown when codex compaction fails (#75364)
• #97974 /btw answers now ride the background-review warm-cache fork (full context, cache-read cost)
• #97937 /btw answers side questions with conversation context; /background retired in favor of /bg
• #97921 feat(todo): task lists now nest — subtasks via a ~45-token parent field
• #97930 feat(system_prompt): two-line conversation clock — anchored start + rebuild-day line (salvages #96224)
• #97926 feat(prompt): DEFAULT_AGENT_IDENTITY — behavior spec replaces trait list (concise-by-default)
• #97918 fix(prompt): skills section — de-dupe hermes-agent teaching, presence-gate the skill pointer, drop '(mandatory)'
• #97899 refactor(prompt): platform-hint diet — 7 heavies compressed, −657 tok across the map (facts probe-pinned)
• #97872 fix(skills): grill-me keeps its name (reverts #97831 rename, keeps content upgrade)
• #97833 feat(optional-skills): decision-questionnaire — async questionnaires for blocked decisions
• #97832 feat(optional-skills): setup-wizard-generator — bash wizards for human-only setup steps
• #97831 feat(skills): grill-me becomes plan-interrogation with frontier-rounds interviewing
• #80708 feat(skills): add optional grill-me skill — adversarial plan interview (salvages #43694)
• #97760 refactor(prompt): memory/skills guidance — one builder, positive posture, session-accurate wording (537 → 255 served, −282/call)
• #97650 fix(skill_manage): missing-old_string errors teach recovery instead of dead-ending into write_file clobbers (#33064, salvage #63879)
• #97695 fix(agent): count only substantive auxiliary progress (#96667 salvage)
• #97451 fix(skill_manage): batch failures keep the failing op's file_preview — models no longer recover blind

Desktop & UI

• #96880 fix(desktop): reserve space for pane tab close button
• #97850 refactor(prompt): desktop platform hint — recipe-first widget teaching, source-verified (442 → 307 tok/call)
• #97829 fix(desktop): ::preview inline widgets work on remote/URL connections
• #97816 feat(desktop): Download button on preview file cards
• #97812 fix(desktop): MEDIA: any-file delivery — preview card + Download button for non-media files; guidance aligned
• #97659 refactor(desktop-tools): consolidate preview + project, diet the desktop_ui suite (3,861 → 2,293 tok/call, −41%)
• #97704 fix: P0 batch — five prompt-cache fixes + two desktop persistence fixes (salvage of #97618 #96755 #97327 #96768 #96608 #95886)
• #97705 fix(desktop): HUD clicks, links, and right-click on Linux X11
• #97666 feat(cli): Ghostty-level pets in the interactive CLI
• #97640 fix(desktop): board switcher crashed on every render
• #97636 Kanban board export / import, plus board rename and delete in the desktop switcher

Gateway, channels & integrations

• #97957 fix(relay): Slack force-on unfurl for streamed finals — fresh stamped post + additive delete op cleanup
• #97932 fix(gateway): make /busy available on gateway platforms (salvage of #97875, supersedes #18366)
• #97873 fix(prompt)+feat(gateway): platform-hint truth pass + universal voice-bubble transcode (all 22 hints source-verified)
• #92318 fix(gateway): bound the hygiene-compression turn-hold (salvage of #90845)
• #63755 fix(agent): normalize list-shaped streaming content deltas
• #97677 fix(teams): authenticate Bot Framework connector attachment downloads (pasted images 401)
• #97458 fix(mcp-oauth): avoid locking across resource requests

Core runtime, state & reliability

• #97929 fix(update): unrelated transitional Windows services no longer abort updates (salvage of #96381)
• #97924 fix(skills): a failed batch rollback never destroys the skill or its snapshots (salvage of #97715)
• #97676 perf(state): route 39 pure-read SessionDB methods off the writer lock + AST gate

Maintenance, documentation & formatting

• #97912 chore: map provider-salvage contributor emails (Neel49, amrrs)
• #97946 fmt(js): npm run fix auto-fix
• #97896 fmt(js): npm run fix auto-fix
• #71419 docs(user-stories): add 65 new community stories, newest first
• #97713 fmt(js): npm run fix auto-fix
• #97706 fmt(js): npm run fix auto-fix
• #97642 fmt(js): npm run fix auto-fix
• #97638 fmt(js): npm run fix auto-fix

Source

• GitHub merged-PR search for August 29, 2026

LAST UPDATED: August 23, 2026

Compiled from ~30 r/hermesagent threads and hundreds of comments (Aug 1–23), Hacker News discussion threads, live OpenRouter pricing (pulled Aug 23), and independent benchmark data. Every price is the live API rate unless marked as user-reported. Community disagreements are kept side-by-side on purpose. Companion to u/carlonox's request in the Megathread Submission Request thread — yes, this replaces the stale recommendations.

Part 1: TL;DR — The Consensus Table

The August 2026 earthquake in one paragraph: DeepSeek raised prices Aug 16 (peak/off-peak, up to 3–5× at peak), OpenRouter got bought by Stripe for $7B+, GPT-5.6 Sol dropped 50% on OpenRouter only while Luna (~80% cut) and Terra (20%) got their own cuts, Gemini killed postpay billing Aug 13, and Qwen3.8-27B landed as a hosted option for those without the VRAM ("matches GPT-5.6 Luna on Max reasoning… for the cost of electricity" — one local runner's take). Three of those five change everyone's math.

Part 2: The Decision Tree

Start here, follow the first line that matches you:

• "I spend under $10/mo and want it to stay that way" → DeepSeek V4 Flash direct + aggressive caching (off-peak hours). Below even that: OpenRouter free tier or NVIDIA NIM, accepting rate limits.
• "I have a $20/mo budget and want zero admin" → Ollama Cloud Pro if privacy (zero-retention) matters most; ChatGPT Plus (Luna) if you're already in that ecosystem; Nous Portal Plus for 300+ models on one key with Hermes-native integration.
• "I run agents all day and pay-as-you-go is eating me alive" → Flat subscription, full stop. As one poster put it: "Agents love getting stuck in retry loops and vaporizing pay-as-you-go balances while you sleep, so locking in a flat monthly sub is just basic financial self-defense." Codex $200 OAuth is the current king of raw tokens (1.4B/mo reported, ~90% cache hits).
• "I need maximum quality on hard tasks regardless of cost" → Claude Opus 5 or Sonnet 5 via direct API, or GPT-5.6 Sol at high effort. Note: Opus-class is the bar everyone measures against, but one camp now argues GLM-5.3 tied Opus 5 on the agentic index at a fraction of the price.
• "I want to orchestrate multi-agent Hermes setups" → The emerging community pattern: a strong orchestrator + cheap subagents. Reported working combos: GLM orchestrating + DeepSeek Flash subagents; Terra-high default with Sol reserved for stuck-points.
• "I'm price-sensitive but don't trust Chinese data handling" → This disqualifies DeepSeek/Z.ai/Kimi/MiniMax direct. Your lane: Ollama Cloud (zero retention), Gemini free tier (mind the key-leak hygiene), or Western-hosted open models via DeepInfra/Fireworks — noting these still serve Chinese-origin weights, so the distinction is about who operates the endpoint. Free tiers generally mean your data trains something.

Part 3: Provider Tiers

🥇 Tier 1 — The community defaults

DeepSeek direct API — still the price floor off-peak for most users, superb tool-calling, cache-hit rates up to 97% reported. Dissenting camp exists post-hike: some argue the peak/off-peak complexity means "not worth it via direct API anymore" since gateways kept old prices — weigh whether you'll actually hit off-peak windows. The Aug 16 hike introduced peak/off-peak windows (peak = 01:00–04:00 & 06:00–10:00 UTC; weekends all-off-peak). Budget win reported: $5.80 over 3 weeks with caching discipline [user-reported]. Watch-for: no vision; your data may train their models.

OpenRouter — 500+ models, one key, auto-fallback, widely praised developer experience (though one HN commenter reports smart routing producing worse results at higher cost). Now owned by Stripe ($7B+, closed Aug 19). Sentiment split: praised for UX, panned for support ("No support exists when things go wrong") and post-acquisition anxiety ("RIP OpenRouter… enshrittification will be inevitable" — one voice among many, not consensus). Critical operational tip from the burn threads: pin to ONE provider per model. Provider-mixing destroys prompt-cache hits and can double or triple your effective cost.

ChatGPT/Codex subscriptions — Luna at $20 is described as "practically unlimited" and needs ~2.8× fewer tokens than DS Flash for similar task performance. Codex $200 = 20× monthly tokens via OAuth. Watch-for: context caps (~256k–272k) on subscription tiers.

🥈 Tier 2 — Strong alternatives

Nous Portal ($20/mo Plus) — the official Hermes-native path: 300+ models, one subscription, tool gateway, 900k context added Aug 19. Verdicts are genuinely split: fans report 50M tokens ≈ $2 during promos; critics calculate only "$22 in API value" for the base plan vs Codex's "$70–100" [anecdotal]. One user-reported burn: ~€70/week through Portal API credits, mostly DS Flash 0731 [anecdotal]. DeepSeek V4 Flash notably did NOT raise prices there after Aug 16. If you want the zero-config Hermes experience, this is the path of least resistance.

Ollama Cloud Pro ($20) — the privacy pick: zero data-retention policy, DS V4 Flash included. One user claims 15k+ requests/month [anecdotal]; another disputes the weekly-vs-monthly framing. Either way: rare billing complaints.

OpenCode Go/Zen ($10 + credits) — 2× credit value, Hermes-fallback-friendly. Turbulent August: DeepSeek moved to a China-based provider (ZDR concerns), the 0813 model was briefly broken during pricing rollout, and rate-limit complaints surfaced. They've said they're working on matching old prices with own inference. Watch this space before committing.

Z.ai (GLM-5.3) — tied #1 agentic index with Opus 5 (HN, Aug 18 release). The Max coding plan (~$120/mo) gets called "the best/most affordable plan" for heavy users. Watch-for: Beijing-hours congestion makes Hermes unresponsive during EU/US mornings; API pricing for 5.3 took days to appear after launch (now live at $1.4/$4.4).

DeepInfra — cheap, good caching ("handles caching much better… saves a ton on reasoning via cache reads"). One quality complaint when accessed through OpenRouter pairing. Worth benchmarking yourself.

🥉 Tier 3 — Budget/free lanes (all with catches)

⚠️ Gray-market resellers (discounted token sites, "unlimited Claude" bridges) exist and some users report reliability. These carry real account-ban and payment risk. Anthropic has both sued over subscription advertising and sent legal letters to harness makers bridging subs into APIs. That's your risk landscape; this megathread doesn't recommend them.

Part 4: Model-by-Model Quick Verdicts (cloud)

Part 5: The Cost Reality Section (numbers as reported, single-source flagged)

Horror stories (learn from them):

• $5 in ONE day on DS Flash via OpenRouter — root causes found in-thread: provider-mixing killing cache hits + Hermes skill bloat re-sent every cron session
• $200+ credits burned on "simple coding tasks" via OpenRouter; the poster switched to subscription OAuth afterward
• ChatGPT business plan gone in 2 days — consensus in-thread: Sol was the culprit, not Hermes
• $25/day Gemini 3 Flash for two users + ops [anecdotal]
• Key-leak cautionary tales outside Hermes: $82k in 48h from one leaked Gemini key

Budget wins (also real):

• DS Flash direct with caching discipline: $5.80 over 3 weeks including 140M-token sessions
• Codex $200 OAuth: 1.4B tokens/month, never limit-hit
• MiniMax M3: full-time coding under $1/day
• Nous Portal promo: 50M tokens for ~$2
• 2.5B DeepSeek tokens for $40

The three habits that separate the two columns: pin providers, cache aggressively, cap your cron sessions' skill payloads.

Part 6: FAQ

1. Q: Is OpenRouter still safe to use after the Stripe acquisition? A: Yes today; service continues normally. Long-term pricing/support direction is the open question — the community is watching, not fleeing.

2. Q: Why did my DeepSeek bill triple? A: Aug 16 peak/off-peak pricing. Peak is 01:00–04:00 & 06:00–10:00 UTC daily; weekends are all off-peak. Shift cron jobs to off-peak and check your cache-hit rate.

3. Q: Can I use my Claude Max subscription inside Hermes? A: Not directly — Anthropic subscription tokens don't feed third-party harnesses, and bridge workarounds carry ban risk (documented bans exist). Pay API rates or pick a different provider.

4. Q: What's actually the cheapest way to run Hermes agents all day? A: A flat subscription (Ollama Cloud $20, Nous Portal $20, Codex $200 for serious volume) beats pay-as-you-go for sustained use. Pay-as-you-go wins only with >95% cache-hit discipline on direct DeepSeek.

5. Q: Which model should orchestrate vs execute in multi-agent setups? A: Community pattern: strong-but-pricier orchestrator (GLM-5.3, Terra high), cheap executors (DS Flash, MiniMax) on subagents.

6. Q: Are the free tiers usable for real work? A: For testing and light use, yes. Sustained agent loops hit rate limits fast everywhere, and free tiers generally mean your data trains something.

7. Q: Is Qwen3.8-27B worth running hosted if I can't run it locally? A: It's the cheapest path to its capability class ($0.40/$3). Local runners report it matching Luna-class cloud performance; hosted-endpoint quality varies by provider, so pin a reputable one.

8. Q: Did GPT-5.6 Sol really get cheaper? A: On OpenRouter only (−50% mid-August). Direct OpenAI pricing unchanged — another reason routing choice matters.

Part 7: Knowledge Table

Part 8: Contribute & Correct

Prices moved at least four times this month and will move again — drop corrections with receipts in the comments and I'll fold verified updates into the next revision. If you're a provider rep: disclose that, and bring data. Sources: 30+ r/hermesagent threads from Aug 1–23 and Hacker News discussions were mined during research (community quotes paraphrased with attribution preserved where load-bearing); live OpenRouter API pull Aug 23; Artificial Analysis provider boards. Community quotes are condensed rather than verbatim-linked to keep this readable — ask in comments and I'll drop the permalink for any specific claim.

Not affiliated with any provider listed. No referral links.

Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus independent X benchmarks. Every number is attributed to the poster's stated hardware/runtime/quant. This community contradicts itself on nearly every axis — so this thread keeps the disagreements side-by-side instead of picking a winner for you.

TL;DR

• The consensus pick: a 27B dense multimodal model that genuinely moved the bar for local agentic coding. The strongest claim with controlled evidence behind it isn't benchmarks — it's tool-calling reliability.
• The default ships at xhigh reasoning and it thinks a lot. Low and medium presets score nearly as well on Artificial Analysis (~43/44 intelligence index, within a few points of the xhigh headline) while cutting thinking tokens ~7–9x (and wall time ~6–7x). Most of you should not be running xhigh.
• Knowledge recall regressed vs 3.6 — widely reported and best understood as a deliberate agentic-design tradeoff. Trivia nerds: keep Gemma around.
• Q4_K_M is basically indistinguishable from Q8 on perplexity, but real-world reports split hard below Q6 for complex reasoning. KV cache quantization is one of the most contested settings in the corpus.
• The "neck and neck with DeepSeek V4 / GPT-5.6 Luna Max" AA headline is real but heavily caveated — see the benchmark credibility section before quoting it at your friends.

1. What it's actually good at

Agentic coding (strongest consensus area)

• "Highest level of agency I've ever seen in a local model" (thread): single 3090, Unsloth Q4_K_S + q8 KV, 150k ctx. From one prompt it pulled the OP's class schedule off a convoluted university website via 80 tool calls, zero human intervention.
• 1M+ token run (thread): RTX 5060 Ti 16GB, UD-Q3_K_XL, 73k ctx. Full REST API + MCP server for a legacy forum from 3 prompts.
• Controlled tool-call evidence: in a plain Python tool loop (no framework), one reporter got zero failed calls from 3.8 while Gemma 4 A4B and Qwen3.6 A3B failed often — the same reporter who rates 3.8 below both on raw code quality. Worse judgment, perfect plumbing.

Creative / game generation

• One-shot playable Super Mario clone (Q8, Framework Desktop) — top pushback: "It's in the training data."
• Galaga 1:1 recreation test (UD-Q8_K_XL, 3×3090 + Tesla P40): "This 'Galaga' clone [from 3.6] ended up pretty much being a space invaders clone instead... Qwen 3.8 thinks a LOT, but it draws out those tiny details and absolutely nails it after the fact." A separate r/LocalLLM user one-shot a playable Galaga-style game at IQ4_XS on dual 4060 Tis, and another built an online multiplayer MOBA overnight with an authoritative server and self-play testing.
• Ray-traced spheres in BASIC: 3.8 self-iterates to a correct Cook-Torrance ray-tracer; 3.6 needed hand-holding. Comment: "this feels more like 3.6 to 4.6 than 3.6 to 3.8."

Vision

Works natively (F16 mmproj), including OCR-style reading of a newspaper image at ~1,000 image tokens — but on a 16GB card at 64k ctx + MTP it leaves as little as ~150 MiB VRAM free. Practical advice from the 16GB crowd: keep text-agent and vision profiles separate, or offload the projector (--no-mmproj-offload).

Where it struggles

• Long analytical/document work: "a step backwards" vs 3.6 at default settings — though a legal-domain poster got on-par-with-122B results with MCP + case access. Task-dependent.
• Complex native coding: one failed C kernel effort (6 hours across 3 sessions) [anecdotal], quant unstated; commenters say Q8 minimum for that tier of work.

2. The thinking-level situation (read this before complaining)

xhigh is the shipped default. It is why your context window evaporates.

Measured ladder (RTX 5080 Laptop 16GB, llama.cpp 10451, UD-IQ3_XXS, Q8_0 KV + FA + MTP, pelican-SVG task, 3 seeds):

That's ~6.4x the wall time for +1.5 points on an eyeball task. But on pass/fail SWE-style tasks, xhigh went 9/12 vs 6–7/12 at lower efforts — the premium scales with whether the task has a verifiable failure.

How to change it: --chat-template-kwargs '{"reasoning_effort":"medium"}' (llama.cpp) or the equivalent in LM Studio custom params.

The overthinking debate, both sides preserved:

• Against: "it will do eight or nine web-search turns and spin its wheels down every rabbit hole" (legal work). One reported loop burned 40k+ characters of reasoning on a trivial subtask. One paper-linked post argues intermediate tokens aren't reasoning at all ("Stop Anthropomorphizing Intermediate Tokens," 538 points).
• For: "if the extra thinking produces measurably better results it's actually just the correct amount of thinking." The low/medium AA scores (~43/44) are the strongest counter to "it only wins by overthinking" — though two commenters read that same data in opposite directions.

Practical takeaway from the corpus: medium for chat/analysis, xhigh only when there's a verifiable right answer.

• The strongest controlled effort data of the week is from X: @superalesha's 67-hour, 40-arm run found xhigh burned 7–11× more reasoning tokens than low for 0–4.7 extra points — and in one head-to-head, low matched xhigh exactly (89.3%) at 1/7.5th the tokens. Also: medium scored below low on every stack (all the damage in HumanEval+ — "that preset overthinks short coding tasks"). His verdict: "low is the rational preset. xhigh is for leaderboard screenshots." That's harsher than the Reddit consensus — weigh both, but it's the biggest sample size anyone published this week.

More data points from the week:

• Medium vs xhigh "actually insane" (223 pts): medium ≈ a couple thousand thinking tokens; xhigh 15–20k minimum, one pacman build hit 40k. But the same thread's best counterpoint: on a bug-finding test, xhigh took 7 min vs medium's 80 s and caught every bug; medium only caught the critical ones. And on a research task xhigh autonomously cloned a repo and read source to verify an answer — neither medium nor 3.6 did.
• Different thinking levels (287 pts): "Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning" — the level you pick changes speed, not whether it beats last generation.
• There is no "high" effort — the ladder is low / medium / xhigh(default), and the gap between medium and xhigh is the complaint that keeps generating threads. Commenters note the efforts aren't just prompts: Qwen specifically trained each level's instruction text in during RL.
• Don't confuse budget with effort (PSA): llama.cpp's web-UI reasoning selector is a hard token cap that truncates mid-thought — it is not Qwen's native effort levels, which actually change how thoroughly the model works. On recent builds use --reasoning-effort medium (or the --chat-template-kwargs form on older ones); anything else silently caps instead of steering.
• The "well?" trick: interrupt mid-think and type well? — the model concludes "the user is impatient, let me finish quickly" and wraps up faster. Works, but commenters consider it a last resort; the thinking is where the quality lives.
• Dissenters exist: one medium-vs-xhigh post claiming "1/20th the time for almost the same quality" got pushed back hard — top reply: low/medium left them unimpressed, xhigh is where frontier-tier coding shows up. The honest split: for chat and eyeball tasks medium is ~free; for verifiable correctness xhigh keeps earning its cost.

3. Knowledge regression vs 3.6 — real, and deliberate

The dedicated thread: 3.8 fails pocket-trivia questions 3.6 reliably answered, at every quant tried. AA's offline Omniscience benchmark agrees.

Community framing: 3.8 is trained to go search instead of recalling, i.e., an agent-first tradeoff. Mitigations posted: RAG/MCP (offline Wikipedia ZIM), or run Gemma 4 31B as a knowledge sidecar.

Counter-data point: a separate legal-work thread reports Harvey-benchmark scores on par with Qwen 3.5-122B once MCP + case access are attached (61/75 raw vs 71/75 with a tool backend). The knowledge didn't vanish; it moved into the toolbox.

4. Quants: what holds up

The one controlled perplexity sweep (16GB-fitting quants, wikitext-2, RTX 5060 Ti)

Poster's call: Q4_K_M is the sweet spot; NVFP4 was the biggest disappointment (same size as IQ4_XS, worse PPL). Pushback worth reading: "PPL degrades less than real world performance… ordering flips near the 4-bit level."

The Q4-vs-Q6 war (unresolved)

• Team Q6/Q8: "q8 dramatically better than q4 for complex reasoning"; one user reports flawless 264k-ctx Q6_K_XL sessions, 2 mistakes per 2M tokens.
• Team Q4-fine: "I run q4 and can only praise the model… just do not go below q8 KV cache."
• Nuance: "there are like 5 different Q4s and they are not equal" — NVFP4 ≠ MXFP4 ≠ Q4_0 ≠ UD-Q4_K_XL. Past ~Q5 with dynamic quants, differences get hard to detect.

The biggest controlled quant test of the week (X)

@superalesha ran a 67-hour benchmark: five full production stacks (FP8 vLLM, NVFP4 W4A16 vLLM, AWQ INT4 vLLM, GGUF Q4_K_M llama.cpp, NInfer — all on RTX 3090s), 40 arms across every reasoning effort, 4,800 tasks / 10,120 requests / 14.5M reasoning tokens, no caps. Results:

• At xhigh every quant landed between 88.0–90.0% pass@1 — AWQ INT4 90.0%, NVFP4/GGUF-Q4_K_M 89.3%, FP8 baseline 88.7%, NInfer 88.0%. The 4-bit quants scored above FP8; McNemar says statistical tie (first vs last = 3 tasks out of 150). "The gap between quants is smaller than the gap between reasoning presets."
• The weirdest number: GGUF Q4_K_M at low effort scored the same 89.3% as xhigh — on 86k reasoning tokens instead of 651k. Across all stacks, xhigh burned 7–11× more tokens than low for 0–4.7 points.
• The one statistically real gap: NVFP4 with reasoning OFF collapsed on HumanEval+ (13/30 vs FP8's 30/30, p=0.0041). Flip it to low and it's instantly back to 90/90. Never run reasoning off — it costs 8–12 points everywhere.
• His cheat sheet: max quality = AWQ INT4 xhigh; daily driver = GGUF Q4_K_M low; honesty note: three of his FP8 arms failed his own methodology audit (leftover token caps) and are being rerun.

This largely settles the Q4-vs-Q6 war for this model at task-level benchmarks — but note the tension with the PPL sweep above: perplexity says NVFP4 is measurably worse than IQ4_XS; task performance says they tie. Both can be true (PPL measures token-level divergence; tasks measure whether errors get caught). And community reports of Q4 reasoning loops remain real — "passes benchmarks" and "never loops in a 2M-token session" are different requirements.

1-bit: comedy, not compute

Unsloth founder in the 1-bit thread: "I would not suggest folks use 1-bit for agentic use cases / tool calls" — divergence hits 92% from BF16 by token 32. General chat survives; agents don't. If you must: presence_penalty = 1.5.

KV cache — among the most contested settings in the corpus

• f16-vs-q8_0 are not equivalents per one AMD tester (f16 held quality past 120k ctx).
• But 16GB users run q4_0/q4_1 KV happily at 64k–164k all week.
• Working rule from comments: don't quantize KV unless you must; if you do, aim ≥ q6; word-of-mouth floor is Q4 model + Q8 KV for agent loops.

Unsloth Dynamic v3 notes

MTP removed from quants below UD-Q2_K_XL and re-uploaded separately (some users still see draft logs in Q5_K_XL — unresolved). Imatrix released; no QAT used.

5. Performance matrix (attributed)

Why "~200 tok/s" claims don't reproduce for you: Windows/WDDM costs 10–15% vs Linux; headlines are measured at short contexts; MTP acceptance is workload-dependent (drops on prose, sometimes net-slower); and the fastest figures come from Blackwell-tuned engines (ninfer), not llama.cpp.

X/Twitter benchmark highlights

• @analogalok's full RTX 4090 matrix: UD-Q4_K_XL on latest llama.cpp. FP16 KV tops out at 100k ctx (40.9 t/s); q8 KV reaches 170k; q4_0 KV fits the full 262k native context in 24GB at 40.7 t/s. Native MTP: 59–60 t/s at 80–130k. Includes exact reproduction flags.
• His follow-up: --parallel 1 + a Q2_K DFlash2 drafter unlocks 250k ctx @ 73.7 t/s (Q4 KV), 150k @ 75 t/s (Q8 KV), or 90k @ 80.6 t/s (FP16 KV) on one 4090 (requires llama.cpp PR #27342).
• NVIDIA forums: DGX Spark face-off, SGLang+DFlash2 vs vLLM+MTP, greedy vs official thinking sampler — DFlash2 won.

6. Failure modes & bugs (reproducible ones)

1. Tool-call failures are usually your tool list, not the model. Best controlled experiment in the corpus: 8 undescribed tools → 0/6 successes; the same tool alone → 15/15; 13 described tools mid-list → 0/5, moved to end → 3/3. Give every tool a description, put critical tools last, don't put examples in descriptions. Every framework failure report (Opencode/Pi/Claude Code) has a plain-loop counterexample in the same threads.
2. Hermes harness specifically: constant tool-call failures on vLLM; "perfect, no issues" on llama.cpp --jinja + q8_0 KV at 256k. Template/parser alignment issue, not weights.
3. Hallucinated user instructions during thinking (reproduced on 2 machines, Pi harness): the model imagines an impatient user and once reverted a commit after imagining a French objection. Community fix: the froggeric fixed chat template (see section 7) eliminates the stock-template tool-call/recovery bugs.
4. temp=1.0 garbage output: thinking falls apart into single-character spam within 10–20k tokens across llama.cpp/vLLM, INT4 through BF16. Diagnosis: sampler, not quant. Fixes: temp 0.1, or split sampling (0.8 main / 0.2 post-thinking). Counter-report: temp 0 caused a 70k-token loop instead. No universal setting exists — tune per task.
5. Decode degradation: 122 → 69 t/s within one generation on 5090 llama.cpp; vLLM/ninfer hold >100. Bug filed upstream.
6. Long-context quality drop: an NVFP4+vLLM eval on B200 scored only ~37% correct in its longest context bucket [single report]; separately, a commenter running official BF16/FP8 via the published vLLM recipe reports agents degrading past ~20k tokens and structured outputs breaking past 20k [single report]. Counterpoint: an f16-KV user on UD-Q4_K_XL (ROCm) says their setup held quality past 120k ctx. Config-dependent; verify on yours.
7. Q8 anomaly reports (Unsloth UD_Q8_K_XL offload/CPU pegging): weak evidence, disputed; most Q8 users report zero issues.
8. Reasoning loops at aggressive quants: 40k characters looping on "angry birds" at Q4-with-QKV-quant, including self-aware "I'm stuck in a loop" narration. Never-seen-it-at-Q6 claims abound.

7. The chat-template situation (read before debugging anything)

The official Qwen 3.8 Jinja template shipped with real bugs, and the community shipped fixes within 48 hours:

• Official template issues: enable_thinking=false crashes; multi-turn history gets poisoned with blank \\think tags; tool calls crash when your client sends arguments as JSON strings (the standard OpenAI format); mid-dialogue system messages get dropped, wedging agent loops.
• froggeric/Qwen-Fixed-Chat-Templates (HF, thread, 334 pts) is the consensus drop-in replacement: safe medium default (kills the burn-20k-tokens-then-return-empty xhigh bug), thinking toggle restored, JSON-string tool-call crash fixed, inline effort steering via <|think_low|> / <|think_medium|> / <|think_xhigh|>, and chronological thought preservation for clean KV prefix caching. Actively maintained — v22.1 as of Aug 21.
• Format-fidelity alternative: a second template stays closer to the exact official prompt format on the theory that deviations subtly degrade quality even when they look fine manually. Pick it if you're benchmarking; pick froggeric for daily driving.
• Upstream note: llama.cpp merged reasoning_effort forwarding on Aug 14 — recent builds pass reasoning_effort to any template correctly. That fixes the plumbing, not the official template's own bugs. A fixed template is still recommended.

8. Benchmarks: believe selectively

• Artificial Analysis: headline posts put 3.8-27B neck-and-neck with DeepSeek V4 and GPT-5.6 Luna Max. Low/medium presets score ~43/44 — the key evidence the gains aren't pure overthinking. Agentic index: medium = xhigh − 1 point.
• The pushback ("A meaningless benchmark", 106 points): the index ranks this 27B above DSV4 Pro, Kimi 2.7 Code, Opus 4.6 and Sonnet 5 — "whatever 'Intelligence' means to AA... is definitely not the same definition we should be using here." Defenders: it's an aggregate skewed toward agentic/science/coding; read the methodology and pick sub-benchmarks for your use case. LiveBench gets respect for monthly task refreshes.
• Best independent test found: AIME 2026, exact-match, temp 0, pass@1 — FP8-xhigh scored 29/30 (96.7%), tying Opus 4.6 and DeepSeek V4 Pro in the poster's table, vs 94.1% for Qwen3.6-27B. Caveats: single run, problem 7 exhausted the token budget in both precisions (empty, not wrong).
• Production blind A/B (thousands of tasks): 3.8 wasn't worse at doing the thing — it was worse at knowing when not to do the thing (+50% noise output).
• Honest calibration: "Opus-level" is real at some tasks, with the right quant and harness. The thread titled "Qwen 3.8 isn't Opus 4.6 level. Let's not be silly." failed at Q6 in VS Code — commenters blamed the editor and the quant, but the burden of proof stays on the demo.

9. Ecosystem: what shipped this week

• DFlash2 (llama.cpp PR #27342, still in review): 2.26×–4.68× on real coding prompts, +2.7GB VRAM. N-max 5 beats the recommended 7; --spec-draft-p-min silently does nothing; stacking ngram-mod hurt (opposite of DFlash1 on 3.6).
• ninfer: Blackwell/5090-tuned engine; 120–160 t/s quants; 480 t/s at 4-way concurrency. Likely source of the unreproducible speed screenshots.
• AutoRound INT4 / AWQ-INT4 GGUFs for vLLM serving.
• KVarN 4/2-bit KV ported to vLLM 0.27.1 — 262k fits small cards, needle-test passes at 240k, ~20% slower decode.
• Uncensored/abliterated variants shipped fast: Huihui-ai ablit, an "Uncensored Aggressive" release bundling K_P quants + HauhauCS FastMTP (up to 3.02× TG claimed), and FP8 abliteration reporting refusal rates dropping to 0–6% — with the community counterpoint that the same tables show 30–50% caveat-rate degradation next to those numbers. Quality varies wildly; check benchmark deltas before switching.

What's coming

• 35B-A3B spotted in ms-swift commits (Aug 15). 16GB-card owners are hyped; early numbers suggest ~27–40 t/s on hardware where the dense 27B crawls.
• A new midsize open-weight model "next week (hopefully)" per Qwen's community manager — no early access this cycle. Speculation centers on ~80B with vision.
• The flagship Qwen3.8-2.4T-A95B got day-0 vLLM support with open weights announced at launch; it barely appears in this week's local-community threads beyond speed speculation (a 2.4T open-weight Call of Duty clone demo made rounds). Local discussion is overwhelmingly about the 27B.

Report template (steal this)

So your numbers mean something to the next reader:

Runtime/version:
Hardware:
Model file + quant:
KV cache:
Speculative (MTP/DFlash2/ngram):
Reasoning effort:
Sampling:
Context size:
Prefill tok/s:
Decode tok/s:
Task used:
Compared against:
Observed result:

Megathread compiled Aug 22, 2026 from r/LocalLLaMA and r/LocalLLM (Aug 15–22) plus public X benchmark threads. All performance figures belong to the hardware/runtime that produced them — the corpus contradicts itself on nearly every axis, and in most cases you can name the variable that explains the split.

Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus independent X benchmarks. Every number is attributed to the poster's stated hardware/runtime/quant. This community contradicts itself on nearly every axis — so this thread keeps the disagreements side-by-side instead of picking a winner for you.

TL;DR

• The consensus pick: a 27B dense multimodal model that genuinely moved the bar for local agentic coding. The strongest claim with controlled evidence behind it isn't benchmarks — it's tool-calling reliability.
• The default ships at xhigh reasoning and it thinks a lot. Low and medium presets score nearly as well on Artificial Analysis (~43/44 intelligence index, within a few points of the xhigh headline) while cutting thinking tokens ~7–9x (and wall time ~6–7x). Most of you should not be running xhigh.
• Knowledge recall regressed vs 3.6 — widely reported and best understood as a deliberate agentic-design tradeoff. Trivia nerds: keep Gemma around.
• Q4_K_M is basically indistinguishable from Q8 on perplexity, but real-world reports split hard below Q6 for complex reasoning. KV cache quantization is one of the most contested settings in the corpus.
• The "neck and neck with DeepSeek V4 / GPT-5.6 Luna Max" AA headline is real but heavily caveated — see the benchmark credibility section before quoting it at your friends.

1. What it's actually good at

Agentic coding (strongest consensus area)

• "Highest level of agency I've ever seen in a local model" (thread): single 3090, Unsloth Q4_K_S + q8 KV, 150k ctx. From one prompt it pulled the OP's class schedule off a convoluted university website via 80 tool calls, zero human intervention.
• 1M+ token run (thread): RTX 5060 Ti 16GB, UD-Q3_K_XL, 73k ctx. Full REST API + MCP server for a legacy forum from 3 prompts.
• Controlled tool-call evidence: in a plain Python tool loop (no framework), one reporter got zero failed calls from 3.8 while Gemma 4 A4B and Qwen3.6 A3B failed often — the same reporter who rates 3.8 below both on raw code quality. Worse judgment, perfect plumbing.

Creative / game generation

• One-shot playable Super Mario clone (Q8, Framework Desktop) — top pushback: "It's in the training data."
• Galaga 1:1 recreation test (UD-Q8_K_XL, 3×3090 + Tesla P40): "This 'Galaga' clone [from 3.6] ended up pretty much being a space invaders clone instead... Qwen 3.8 thinks a LOT, but it draws out those tiny details and absolutely nails it after the fact." A separate r/LocalLLM user one-shot a playable Galaga-style game at IQ4_XS on dual 4060 Tis, and another built an online multiplayer MOBA overnight with an authoritative server and self-play testing.
• Ray-traced spheres in BASIC: 3.8 self-iterates to a correct Cook-Torrance ray-tracer; 3.6 needed hand-holding. Comment: "this feels more like 3.6 to 4.6 than 3.6 to 3.8."

Vision

Works natively (F16 mmproj), including OCR-style reading of a newspaper image at ~1,000 image tokens — but on a 16GB card at 64k ctx + MTP it leaves as little as ~150 MiB VRAM free. Practical advice from the 16GB crowd: keep text-agent and vision profiles separate, or offload the projector (--no-mmproj-offload).

Where it struggles

• Long analytical/document work: "a step backwards" vs 3.6 at default settings — though a legal-domain poster got on-par-with-122B results with MCP + case access. Task-dependent.
• Complex native coding: one failed C kernel effort (6 hours across 3 sessions) [anecdotal], quant unstated; commenters say Q8 minimum for that tier of work.

2. The thinking-level situation (read this before complaining)

xhigh is the shipped default. It is why your context window evaporates.

Measured ladder (RTX 5080 Laptop 16GB, llama.cpp 10451, UD-IQ3_XXS, Q8_0 KV + FA + MTP, pelican-SVG task, 3 seeds):

That's ~6.4x the wall time for +1.5 points on an eyeball task. But on pass/fail SWE-style tasks, xhigh went 9/12 vs 6–7/12 at lower efforts — the premium scales with whether the task has a verifiable failure.

How to change it: --chat-template-kwargs '{"reasoning_effort":"medium"}' (llama.cpp) or the equivalent in LM Studio custom params.

The overthinking debate, both sides preserved:

• Against: "it will do eight or nine web-search turns and spin its wheels down every rabbit hole" (legal work). One reported loop burned 40k+ characters of reasoning on a trivial subtask. One paper-linked post argues intermediate tokens aren't reasoning at all ("Stop Anthropomorphizing Intermediate Tokens," 538 points).
• For: "if the extra thinking produces measurably better results it's actually just the correct amount of thinking." The low/medium AA scores (~43/44) are the strongest counter to "it only wins by overthinking" — though two commenters read that same data in opposite directions.

Practical takeaway from the corpus: medium for chat/analysis, xhigh only when there's a verifiable right answer.

• The strongest controlled effort data of the week is from X: @superalesha's 67-hour, 40-arm run found xhigh burned 7–11× more reasoning tokens than low for 0–4.7 extra points — and in one head-to-head, low matched xhigh exactly (89.3%) at 1/7.5th the tokens. Also: medium scored below low on every stack (all the damage in HumanEval+ — "that preset overthinks short coding tasks"). His verdict: "low is the rational preset. xhigh is for leaderboard screenshots." That's harsher than the Reddit consensus — weigh both, but it's the biggest sample size anyone published this week.

More data points from the week:

• Medium vs xhigh "actually insane" (223 pts): medium ≈ a couple thousand thinking tokens; xhigh 15–20k minimum, one pacman build hit 40k. But the same thread's best counterpoint: on a bug-finding test, xhigh took 7 min vs medium's 80 s and caught every bug; medium only caught the critical ones. And on a research task xhigh autonomously cloned a repo and read source to verify an answer — neither medium nor 3.6 did.
• Different thinking levels (287 pts): "Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning" — the level you pick changes speed, not whether it beats last generation.
• There is no "high" effort — the ladder is low / medium / xhigh(default), and the gap between medium and xhigh is the complaint that keeps generating threads. Commenters note the efforts aren't just prompts: Qwen specifically trained each level's instruction text in during RL.
• Don't confuse budget with effort (PSA): llama.cpp's web-UI reasoning selector is a hard token cap that truncates mid-thought — it is not Qwen's native effort levels, which actually change how thoroughly the model works. On recent builds use --reasoning-effort medium (or the --chat-template-kwargs form on older ones); anything else silently caps instead of steering.
• The "well?" trick: interrupt mid-think and type well? — the model concludes "the user is impatient, let me finish quickly" and wraps up faster. Works, but commenters consider it a last resort; the thinking is where the quality lives.
• Dissenters exist: one medium-vs-xhigh post claiming "1/20th the time for almost the same quality" got pushed back hard — top reply: low/medium left them unimpressed, xhigh is where frontier-tier coding shows up. The honest split: for chat and eyeball tasks medium is ~free; for verifiable correctness xhigh keeps earning its cost.

3. Knowledge regression vs 3.6 — real, and deliberate

The dedicated thread: 3.8 fails pocket-trivia questions 3.6 reliably answered, at every quant tried. AA's offline Omniscience benchmark agrees.

Community framing: 3.8 is trained to go search instead of recalling, i.e., an agent-first tradeoff. Mitigations posted: RAG/MCP (offline Wikipedia ZIM), or run Gemma 4 31B as a knowledge sidecar.

Counter-data point: a separate legal-work thread reports Harvey-benchmark scores on par with Qwen 3.5-122B once MCP + case access are attached (61/75 raw vs 71/75 with a tool backend). The knowledge didn't vanish; it moved into the toolbox.

4. Quants: what holds up

The one controlled perplexity sweep (16GB-fitting quants, wikitext-2, RTX 5060 Ti)

Poster's call: Q4_K_M is the sweet spot; NVFP4 was the biggest disappointment (same size as IQ4_XS, worse PPL). Pushback worth reading: "PPL degrades less than real world performance… ordering flips near the 4-bit level."

The Q4-vs-Q6 war (unresolved)

• Team Q6/Q8: "q8 dramatically better than q4 for complex reasoning"; one user reports flawless 264k-ctx Q6_K_XL sessions, 2 mistakes per 2M tokens.
• Team Q4-fine: "I run q4 and can only praise the model… just do not go below q8 KV cache."
• Nuance: "there are like 5 different Q4s and they are not equal" — NVFP4 ≠ MXFP4 ≠ Q4_0 ≠ UD-Q4_K_XL. Past ~Q5 with dynamic quants, differences get hard to detect.

The biggest controlled quant test of the week (X)

@superalesha ran a 67-hour benchmark: five full production stacks (FP8 vLLM, NVFP4 W4A16 vLLM, AWQ INT4 vLLM, GGUF Q4_K_M llama.cpp, NInfer — all on RTX 3090s), 40 arms across every reasoning effort, 4,800 tasks / 10,120 requests / 14.5M reasoning tokens, no caps. Results:

• At xhigh every quant landed between 88.0–90.0% pass@1 — AWQ INT4 90.0%, NVFP4/GGUF-Q4_K_M 89.3%, FP8 baseline 88.7%, NInfer 88.0%. The 4-bit quants scored above FP8; McNemar says statistical tie (first vs last = 3 tasks out of 150). "The gap between quants is smaller than the gap between reasoning presets."
• The weirdest number: GGUF Q4_K_M at low effort scored the same 89.3% as xhigh — on 86k reasoning tokens instead of 651k. Across all stacks, xhigh burned 7–11× more tokens than low for 0–4.7 points.
• The one statistically real gap: NVFP4 with reasoning OFF collapsed on HumanEval+ (13/30 vs FP8's 30/30, p=0.0041). Flip it to low and it's instantly back to 90/90. Never run reasoning off — it costs 8–12 points everywhere.
• His cheat sheet: max quality = AWQ INT4 xhigh; daily driver = GGUF Q4_K_M low; honesty note: three of his FP8 arms failed his own methodology audit (leftover token caps) and are being rerun.

This largely settles the Q4-vs-Q6 war for this model at task-level benchmarks — but note the tension with the PPL sweep above: perplexity says NVFP4 is measurably worse than IQ4_XS; task performance says they tie. Both can be true (PPL measures token-level divergence; tasks measure whether errors get caught). And community reports of Q4 reasoning loops remain real — "passes benchmarks" and "never loops in a 2M-token session" are different requirements.

1-bit: comedy, not compute

Unsloth founder in the 1-bit thread: "I would not suggest folks use 1-bit for agentic use cases / tool calls" — divergence hits 92% from BF16 by token 32. General chat survives; agents don't. If you must: presence_penalty = 1.5.

KV cache — among the most contested settings in the corpus

• f16-vs-q8_0 are not equivalents per one AMD tester (f16 held quality past 120k ctx).
• But 16GB users run q4_0/q4_1 KV happily at 64k–164k all week.
• Working rule from comments: don't quantize KV unless you must; if you do, aim ≥ q6; word-of-mouth floor is Q4 model + Q8 KV for agent loops.

Unsloth Dynamic v3 notes

MTP removed from quants below UD-Q2_K_XL and re-uploaded separately (some users still see draft logs in Q5_K_XL — unresolved). Imatrix released; no QAT used.

5. Performance matrix (attributed)

Why "~200 tok/s" claims don't reproduce for you: Windows/WDDM costs 10–15% vs Linux; headlines are measured at short contexts; MTP acceptance is workload-dependent (drops on prose, sometimes net-slower); and the fastest figures come from Blackwell-tuned engines (ninfer), not llama.cpp.

X/Twitter benchmark highlights

• @analogalok's full RTX 4090 matrix: UD-Q4_K_XL on latest llama.cpp. FP16 KV tops out at 100k ctx (40.9 t/s); q8 KV reaches 170k; q4_0 KV fits the full 262k native context in 24GB at 40.7 t/s. Native MTP: 59–60 t/s at 80–130k. Includes exact reproduction flags.
• His follow-up: --parallel 1 + a Q2_K DFlash2 drafter unlocks 250k ctx @ 73.7 t/s (Q4 KV), 150k @ 75 t/s (Q8 KV), or 90k @ 80.6 t/s (FP16 KV) on one 4090 (requires llama.cpp PR #27342).
• NVIDIA forums: DGX Spark face-off, SGLang+DFlash2 vs vLLM+MTP, greedy vs official thinking sampler — DFlash2 won.

6. Failure modes & bugs (reproducible ones)

1. Tool-call failures are usually your tool list, not the model. Best controlled experiment in the corpus: 8 undescribed tools → 0/6 successes; the same tool alone → 15/15; 13 described tools mid-list → 0/5, moved to end → 3/3. Give every tool a description, put critical tools last, don't put examples in descriptions. Every framework failure report (Opencode/Pi/Claude Code) has a plain-loop counterexample in the same threads.
2. Hermes harness specifically: constant tool-call failures on vLLM; "perfect, no issues" on llama.cpp --jinja + q8_0 KV at 256k. Template/parser alignment issue, not weights.
3. Hallucinated user instructions during thinking (reproduced on 2 machines, Pi harness): the model imagines an impatient user and once reverted a commit after imagining a French objection. Community fix: the froggeric fixed chat template (see section 7) eliminates the stock-template tool-call/recovery bugs.
4. temp=1.0 garbage output: thinking falls apart into single-character spam within 10–20k tokens across llama.cpp/vLLM, INT4 through BF16. Diagnosis: sampler, not quant. Fixes: temp 0.1, or split sampling (0.8 main / 0.2 post-thinking). Counter-report: temp 0 caused a 70k-token loop instead. No universal setting exists — tune per task.
5. Decode degradation: 122 → 69 t/s within one generation on 5090 llama.cpp; vLLM/ninfer hold >100. Bug filed upstream.
6. Long-context quality drop: an NVFP4+vLLM eval on B200 scored only ~37% correct in its longest context bucket [single report]; separately, a commenter running official BF16/FP8 via the published vLLM recipe reports agents degrading past ~20k tokens and structured outputs breaking past 20k [single report]. Counterpoint: an f16-KV user on UD-Q4_K_XL (ROCm) says their setup held quality past 120k ctx. Config-dependent; verify on yours.
7. Q8 anomaly reports (Unsloth UD_Q8_K_XL offload/CPU pegging): weak evidence, disputed; most Q8 users report zero issues.
8. Reasoning loops at aggressive quants: 40k characters looping on "angry birds" at Q4-with-QKV-quant, including self-aware "I'm stuck in a loop" narration. Never-seen-it-at-Q6 claims abound.

7. The chat-template situation (read before debugging anything)

The official Qwen 3.8 Jinja template shipped with real bugs, and the community shipped fixes within 48 hours:

• Official template issues: enable_thinking=false crashes; multi-turn history gets poisoned with blank \\think tags; tool calls crash when your client sends arguments as JSON strings (the standard OpenAI format); mid-dialogue system messages get dropped, wedging agent loops.
• froggeric/Qwen-Fixed-Chat-Templates (HF, thread, 334 pts) is the consensus drop-in replacement: safe medium default (kills the burn-20k-tokens-then-return-empty xhigh bug), thinking toggle restored, JSON-string tool-call crash fixed, inline effort steering via <|think_low|> / <|think_medium|> / <|think_xhigh|>, and chronological thought preservation for clean KV prefix caching. Actively maintained — v22.1 as of Aug 21.
• Format-fidelity alternative: a second template stays closer to the exact official prompt format on the theory that deviations subtly degrade quality even when they look fine manually. Pick it if you're benchmarking; pick froggeric for daily driving.
• Upstream note: llama.cpp merged reasoning_effort forwarding on Aug 14 — recent builds pass reasoning_effort to any template correctly. That fixes the plumbing, not the official template's own bugs. A fixed template is still recommended.

8. Benchmarks: believe selectively

• Artificial Analysis: headline posts put 3.8-27B neck-and-neck with DeepSeek V4 and GPT-5.6 Luna Max. Low/medium presets score ~43/44 — the key evidence the gains aren't pure overthinking. Agentic index: medium = xhigh − 1 point.
• The pushback ("A meaningless benchmark", 106 points): the index ranks this 27B above DSV4 Pro, Kimi 2.7 Code, Opus 4.6 and Sonnet 5 — "whatever 'Intelligence' means to AA... is definitely not the same definition we should be using here." Defenders: it's an aggregate skewed toward agentic/science/coding; read the methodology and pick sub-benchmarks for your use case. LiveBench gets respect for monthly task refreshes.
• Best independent test found: AIME 2026, exact-match, temp 0, pass@1 — FP8-xhigh scored 29/30 (96.7%), tying Opus 4.6 and DeepSeek V4 Pro in the poster's table, vs 94.1% for Qwen3.6-27B. Caveats: single run, problem 7 exhausted the token budget in both precisions (empty, not wrong).
• Production blind A/B (thousands of tasks): 3.8 wasn't worse at doing the thing — it was worse at knowing when not to do the thing (+50% noise output).
• Honest calibration: "Opus-level" is real at some tasks, with the right quant and harness. The thread titled "Qwen 3.8 isn't Opus 4.6 level. Let's not be silly." failed at Q6 in VS Code — commenters blamed the editor and the quant, but the burden of proof stays on the demo.

9. Ecosystem: what shipped this week

• DFlash2 (llama.cpp PR #27342, still in review): 2.26×–4.68× on real coding prompts, +2.7GB VRAM. N-max 5 beats the recommended 7; --spec-draft-p-min silently does nothing; stacking ngram-mod hurt (opposite of DFlash1 on 3.6).
• ninfer: Blackwell/5090-tuned engine; 120–160 t/s quants; 480 t/s at 4-way concurrency. Likely source of the unreproducible speed screenshots.
• AutoRound INT4 / AWQ-INT4 GGUFs for vLLM serving.
• KVarN 4/2-bit KV ported to vLLM 0.27.1 — 262k fits small cards, needle-test passes at 240k, ~20% slower decode.
• Uncensored/abliterated variants shipped fast: Huihui-ai ablit, an "Uncensored Aggressive" release bundling K_P quants + HauhauCS FastMTP (up to 3.02× TG claimed), and FP8 abliteration reporting refusal rates dropping to 0–6% — with the community counterpoint that the same tables show 30–50% caveat-rate degradation next to those numbers. Quality varies wildly; check benchmark deltas before switching.

What's coming

• 35B-A3B spotted in ms-swift commits (Aug 15). 16GB-card owners are hyped; early numbers suggest ~27–40 t/s on hardware where the dense 27B crawls.
• A new midsize open-weight model "next week (hopefully)" per Qwen's community manager — no early access this cycle. Speculation centers on ~80B with vision.
• The flagship Qwen3.8-2.4T-A95B got day-0 vLLM support with open weights announced at launch; it barely appears in this week's local-community threads beyond speed speculation (a 2.4T open-weight Call of Duty clone demo made rounds). Local discussion is overwhelmingly about the 27B.

Report template (steal this)

So your numbers mean something to the next reader:

Runtime/version:
Hardware:
Model file + quant:
KV cache:
Speculative (MTP/DFlash2/ngram):
Reasoning effort:
Sampling:
Context size:
Prefill tok/s:
Decode tok/s:
Task used:
Compared against:
Observed result:

Megathread compiled Aug 22, 2026 from r/LocalLLaMA and r/LocalLLM (Aug 15–22) plus public X benchmark threads. All performance figures belong to the hardware/runtime that produced them — the corpus contradicts itself on nearly every axis, and in most cases you can name the variable that explains the split.

Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus independent X benchmarks. Every number is attributed to the poster's stated hardware/runtime/quant. This community contradicts itself on nearly every axis — so this thread keeps the disagreements side-by-side instead of picking a winner for you.

TL;DR

• The consensus pick: a 27B dense multimodal model that genuinely moved the bar for local agentic coding. The strongest claim with controlled evidence behind it isn't benchmarks — it's tool-calling reliability.
• The default ships at xhigh reasoning and it thinks a lot. Low and medium presets score nearly as well on Artificial Analysis (~43/44 intelligence index, within a few points of the xhigh headline) while cutting thinking tokens ~7–9x (and wall time ~6–7x). Most of you should not be running xhigh.
• Knowledge recall regressed vs 3.6 — widely reported and best understood as a deliberate agentic-design tradeoff. Trivia nerds: keep Gemma around.
• Q4_K_M is basically indistinguishable from Q8 on perplexity, but real-world reports split hard below Q6 for complex reasoning. KV cache quantization is one of the most contested settings in the corpus.
• The "neck and neck with DeepSeek V4 / GPT-5.6 Luna Max" AA headline is real but heavily caveated — see the benchmark credibility section before quoting it at your friends.

1. What it's actually good at

Agentic coding (strongest consensus area)

• "Highest level of agency I've ever seen in a local model" (thread): single 3090, Unsloth Q4_K_S + q8 KV, 150k ctx. From one prompt it pulled the OP's class schedule off a convoluted university website via 80 tool calls, zero human intervention.
• 1M+ token run (thread): RTX 5060 Ti 16GB, UD-Q3_K_XL, 73k ctx. Full REST API + MCP server for a legacy forum from 3 prompts.
• Controlled tool-call evidence: in a plain Python tool loop (no framework), one reporter got zero failed calls from 3.8 while Gemma 4 A4B and Qwen3.6 A3B failed often — the same reporter who rates 3.8 below both on raw code quality. Worse judgment, perfect plumbing.

Creative / game generation

• One-shot playable Super Mario clone (Q8, Framework Desktop) — top pushback: "It's in the training data."
• Galaga 1:1 recreation test (UD-Q8_K_XL, 3×3090 + Tesla P40): "This 'Galaga' clone [from 3.6] ended up pretty much being a space invaders clone instead... Qwen 3.8 thinks a LOT, but it draws out those tiny details and absolutely nails it after the fact." A separate r/LocalLLM user one-shot a playable Galaga-style game at IQ4_XS on dual 4060 Tis, and another built an online multiplayer MOBA overnight with an authoritative server and self-play testing.
• Ray-traced spheres in BASIC: 3.8 self-iterates to a correct Cook-Torrance ray-tracer; 3.6 needed hand-holding. Comment: "this feels more like 3.6 to 4.6 than 3.6 to 3.8."

Vision

Works natively (F16 mmproj), including OCR-style reading of a newspaper image at ~1,000 image tokens — but on a 16GB card at 64k ctx + MTP it leaves as little as ~150 MiB VRAM free. Practical advice from the 16GB crowd: keep text-agent and vision profiles separate, or offload the projector (--no-mmproj-offload).

Where it struggles

• Long analytical/document work: "a step backwards" vs 3.6 at default settings — though a legal-domain poster got on-par-with-122B results with MCP + case access. Task-dependent.
• Complex native coding: one failed C kernel effort (6 hours across 3 sessions) [anecdotal], quant unstated; commenters say Q8 minimum for that tier of work.

2. The thinking-level situation (read this before complaining)

xhigh is the shipped default. It is why your context window evaporates.

Measured ladder (RTX 5080 Laptop 16GB, llama.cpp 10451, UD-IQ3_XXS, Q8_0 KV + FA + MTP, pelican-SVG task, 3 seeds):

That's ~6.4x the wall time for +1.5 points on an eyeball task. But on pass/fail SWE-style tasks, xhigh went 9/12 vs 6–7/12 at lower efforts — the premium scales with whether the task has a verifiable failure.

How to change it: --chat-template-kwargs '{"reasoning_effort":"medium"}' (llama.cpp) or the equivalent in LM Studio custom params.

The overthinking debate, both sides preserved:

• Against: "it will do eight or nine web-search turns and spin its wheels down every rabbit hole" (legal work). One reported loop burned 40k+ characters of reasoning on a trivial subtask. One paper-linked post argues intermediate tokens aren't reasoning at all ("Stop Anthropomorphizing Intermediate Tokens," 538 points).
• For: "if the extra thinking produces measurably better results it's actually just the correct amount of thinking." The low/medium AA scores (~43/44) are the strongest counter to "it only wins by overthinking" — though two commenters read that same data in opposite directions.

Practical takeaway from the corpus: medium for chat/analysis, xhigh only when there's a verifiable right answer.

• The strongest controlled effort data of the week is from X: @superalesha's 67-hour, 40-arm run found xhigh burned 7–11× more reasoning tokens than low for 0–4.7 extra points — and in one head-to-head, low matched xhigh exactly (89.3%) at 1/7.5th the tokens. Also: medium scored below low on every stack (all the damage in HumanEval+ — "that preset overthinks short coding tasks"). His verdict: "low is the rational preset. xhigh is for leaderboard screenshots." That's harsher than the Reddit consensus — weigh both, but it's the biggest sample size anyone published this week.

More data points from the week:

• Medium vs xhigh "actually insane" (223 pts): medium ≈ a couple thousand thinking tokens; xhigh 15–20k minimum, one pacman build hit 40k. But the same thread's best counterpoint: on a bug-finding test, xhigh took 7 min vs medium's 80 s and caught every bug; medium only caught the critical ones. And on a research task xhigh autonomously cloned a repo and read source to verify an answer — neither medium nor 3.6 did.
• Different thinking levels (287 pts): "Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning" — the level you pick changes speed, not whether it beats last generation.
• There is no "high" effort — the ladder is low / medium / xhigh(default), and the gap between medium and xhigh is the complaint that keeps generating threads. Commenters note the efforts aren't just prompts: Qwen specifically trained each level's instruction text in during RL.
• Don't confuse budget with effort (PSA): llama.cpp's web-UI reasoning selector is a hard token cap that truncates mid-thought — it is not Qwen's native effort levels, which actually change how thoroughly the model works. On recent builds use --reasoning-effort medium (or the --chat-template-kwargs form on older ones); anything else silently caps instead of steering.
• The "well?" trick: interrupt mid-think and type well? — the model concludes "the user is impatient, let me finish quickly" and wraps up faster. Works, but commenters consider it a last resort; the thinking is where the quality lives.
• Dissenters exist: one medium-vs-xhigh post claiming "1/20th the time for almost the same quality" got pushed back hard — top reply: low/medium left them unimpressed, xhigh is where frontier-tier coding shows up. The honest split: for chat and eyeball tasks medium is ~free; for verifiable correctness xhigh keeps earning its cost.

3. Knowledge regression vs 3.6 — real, and deliberate

The dedicated thread: 3.8 fails pocket-trivia questions 3.6 reliably answered, at every quant tried. AA's offline Omniscience benchmark agrees.

Community framing: 3.8 is trained to go search instead of recalling, i.e., an agent-first tradeoff. Mitigations posted: RAG/MCP (offline Wikipedia ZIM), or run Gemma 4 31B as a knowledge sidecar.

Counter-data point: a separate legal-work thread reports Harvey-benchmark scores on par with Qwen 3.5-122B once MCP + case access are attached (61/75 raw vs 71/75 with a tool backend). The knowledge didn't vanish; it moved into the toolbox.

4. Quants: what holds up

The one controlled perplexity sweep (16GB-fitting quants, wikitext-2, RTX 5060 Ti)

Poster's call: Q4_K_M is the sweet spot; NVFP4 was the biggest disappointment (same size as IQ4_XS, worse PPL). Pushback worth reading: "PPL degrades less than real world performance… ordering flips near the 4-bit level."

The Q4-vs-Q6 war (unresolved)

• Team Q6/Q8: "q8 dramatically better than q4 for complex reasoning"; one user reports flawless 264k-ctx Q6_K_XL sessions, 2 mistakes per 2M tokens.
• Team Q4-fine: "I run q4 and can only praise the model… just do not go below q8 KV cache."
• Nuance: "there are like 5 different Q4s and they are not equal" — NVFP4 ≠ MXFP4 ≠ Q4_0 ≠ UD-Q4_K_XL. Past ~Q5 with dynamic quants, differences get hard to detect.

The biggest controlled quant test of the week (X)

@superalesha ran a 67-hour benchmark: five full production stacks (FP8 vLLM, NVFP4 W4A16 vLLM, AWQ INT4 vLLM, GGUF Q4_K_M llama.cpp, NInfer — all on RTX 3090s), 40 arms across every reasoning effort, 4,800 tasks / 10,120 requests / 14.5M reasoning tokens, no caps. Results:

• At xhigh every quant landed between 88.0–90.0% pass@1 — AWQ INT4 90.0%, NVFP4/GGUF-Q4_K_M 89.3%, FP8 baseline 88.7%, NInfer 88.0%. The 4-bit quants scored above FP8; McNemar says statistical tie (first vs last = 3 tasks out of 150). "The gap between quants is smaller than the gap between reasoning presets."
• The weirdest number: GGUF Q4_K_M at low effort scored the same 89.3% as xhigh — on 86k reasoning tokens instead of 651k. Across all stacks, xhigh burned 7–11× more tokens than low for 0–4.7 points.
• The one statistically real gap: NVFP4 with reasoning OFF collapsed on HumanEval+ (13/30 vs FP8's 30/30, p=0.0041). Flip it to low and it's instantly back to 90/90. Never run reasoning off — it costs 8–12 points everywhere.
• His cheat sheet: max quality = AWQ INT4 xhigh; daily driver = GGUF Q4_K_M low; honesty note: three of his FP8 arms failed his own methodology audit (leftover token caps) and are being rerun.

This largely settles the Q4-vs-Q6 war for this model at task-level benchmarks — but note the tension with the PPL sweep above: perplexity says NVFP4 is measurably worse than IQ4_XS; task performance says they tie. Both can be true (PPL measures token-level divergence; tasks measure whether errors get caught). And community reports of Q4 reasoning loops remain real — "passes benchmarks" and "never loops in a 2M-token session" are different requirements.

1-bit: comedy, not compute

Unsloth founder in the 1-bit thread: "I would not suggest folks use 1-bit for agentic use cases / tool calls" — divergence hits 92% from BF16 by token 32. General chat survives; agents don't. If you must: presence_penalty = 1.5.

KV cache — among the most contested settings in the corpus

• f16-vs-q8_0 are not equivalents per one AMD tester (f16 held quality past 120k ctx).
• But 16GB users run q4_0/q4_1 KV happily at 64k–164k all week.
• Working rule from comments: don't quantize KV unless you must; if you do, aim ≥ q6; word-of-mouth floor is Q4 model + Q8 KV for agent loops.

Unsloth Dynamic v3 notes

MTP removed from quants below UD-Q2_K_XL and re-uploaded separately (some users still see draft logs in Q5_K_XL — unresolved). Imatrix released; no QAT used.

5. Performance matrix (attributed)

Why "~200 tok/s" claims don't reproduce for you: Windows/WDDM costs 10–15% vs Linux; headlines are measured at short contexts; MTP acceptance is workload-dependent (drops on prose, sometimes net-slower); and the fastest figures come from Blackwell-tuned engines (ninfer), not llama.cpp.

X/Twitter benchmark highlights

• @analogalok's full RTX 4090 matrix: UD-Q4_K_XL on latest llama.cpp. FP16 KV tops out at 100k ctx (40.9 t/s); q8 KV reaches 170k; q4_0 KV fits the full 262k native context in 24GB at 40.7 t/s. Native MTP: 59–60 t/s at 80–130k. Includes exact reproduction flags.
• His follow-up: --parallel 1 + a Q2_K DFlash2 drafter unlocks 250k ctx @ 73.7 t/s (Q4 KV), 150k @ 75 t/s (Q8 KV), or 90k @ 80.6 t/s (FP16 KV) on one 4090 (requires llama.cpp PR #27342).
• NVIDIA forums: DGX Spark face-off, SGLang+DFlash2 vs vLLM+MTP, greedy vs official thinking sampler — DFlash2 won.

6. Failure modes & bugs (reproducible ones)

1. Tool-call failures are usually your tool list, not the model. Best controlled experiment in the corpus: 8 undescribed tools → 0/6 successes; the same tool alone → 15/15; 13 described tools mid-list → 0/5, moved to end → 3/3. Give every tool a description, put critical tools last, don't put examples in descriptions. Every framework failure report (Opencode/Pi/Claude Code) has a plain-loop counterexample in the same threads.
2. Hermes harness specifically: constant tool-call failures on vLLM; "perfect, no issues" on llama.cpp --jinja + q8_0 KV at 256k. Template/parser alignment issue, not weights.
3. Hallucinated user instructions during thinking (reproduced on 2 machines, Pi harness): the model imagines an impatient user and once reverted a commit after imagining a French objection. Community fix: the froggeric fixed chat template (see section 7) eliminates the stock-template tool-call/recovery bugs.
4. temp=1.0 garbage output: thinking falls apart into single-character spam within 10–20k tokens across llama.cpp/vLLM, INT4 through BF16. Diagnosis: sampler, not quant. Fixes: temp 0.1, or split sampling (0.8 main / 0.2 post-thinking). Counter-report: temp 0 caused a 70k-token loop instead. No universal setting exists — tune per task.
5. Decode degradation: 122 → 69 t/s within one generation on 5090 llama.cpp; vLLM/ninfer hold >100. Bug filed upstream.
6. Long-context quality drop: an NVFP4+vLLM eval on B200 scored only ~37% correct in its longest context bucket [single report]; separately, a commenter running official BF16/FP8 via the published vLLM recipe reports agents degrading past ~20k tokens and structured outputs breaking past 20k [single report]. Counterpoint: an f16-KV user on UD-Q4_K_XL (ROCm) says their setup held quality past 120k ctx. Config-dependent; verify on yours.
7. Q8 anomaly reports (Unsloth UD_Q8_K_XL offload/CPU pegging): weak evidence, disputed; most Q8 users report zero issues.
8. Reasoning loops at aggressive quants: 40k characters looping on "angry birds" at Q4-with-QKV-quant, including self-aware "I'm stuck in a loop" narration. Never-seen-it-at-Q6 claims abound.

7. The chat-template situation (read before debugging anything)

The official Qwen 3.8 Jinja template shipped with real bugs, and the community shipped fixes within 48 hours:

• Official template issues: enable_thinking=false crashes; multi-turn history gets poisoned with blank \\think tags; tool calls crash when your client sends arguments as JSON strings (the standard OpenAI format); mid-dialogue system messages get dropped, wedging agent loops.
• froggeric/Qwen-Fixed-Chat-Templates (HF, thread, 334 pts) is the consensus drop-in replacement: safe medium default (kills the burn-20k-tokens-then-return-empty xhigh bug), thinking toggle restored, JSON-string tool-call crash fixed, inline effort steering via <|think_low|> / <|think_medium|> / <|think_xhigh|>, and chronological thought preservation for clean KV prefix caching. Actively maintained — v22.1 as of Aug 21.
• Format-fidelity alternative: a second template stays closer to the exact official prompt format on the theory that deviations subtly degrade quality even when they look fine manually. Pick it if you're benchmarking; pick froggeric for daily driving.
• Upstream note: llama.cpp merged reasoning_effort forwarding on Aug 14 — recent builds pass reasoning_effort to any template correctly. That fixes the plumbing, not the official template's own bugs. A fixed template is still recommended.

8. Benchmarks: believe selectively

• Artificial Analysis: headline posts put 3.8-27B neck-and-neck with DeepSeek V4 and GPT-5.6 Luna Max. Low/medium presets score ~43/44 — the key evidence the gains aren't pure overthinking. Agentic index: medium = xhigh − 1 point.
• The pushback ("A meaningless benchmark", 106 points): the index ranks this 27B above DSV4 Pro, Kimi 2.7 Code, Opus 4.6 and Sonnet 5 — "whatever 'Intelligence' means to AA... is definitely not the same definition we should be using here." Defenders: it's an aggregate skewed toward agentic/science/coding; read the methodology and pick sub-benchmarks for your use case. LiveBench gets respect for monthly task refreshes.
• Best independent test found: AIME 2026, exact-match, temp 0, pass@1 — FP8-xhigh scored 29/30 (96.7%), tying Opus 4.6 and DeepSeek V4 Pro in the poster's table, vs 94.1% for Qwen3.6-27B. Caveats: single run, problem 7 exhausted the token budget in both precisions (empty, not wrong).
• Production blind A/B (thousands of tasks): 3.8 wasn't worse at doing the thing — it was worse at knowing when not to do the thing (+50% noise output).
• Honest calibration: "Opus-level" is real at some tasks, with the right quant and harness. The thread titled "Qwen 3.8 isn't Opus 4.6 level. Let's not be silly." failed at Q6 in VS Code — commenters blamed the editor and the quant, but the burden of proof stays on the demo.

9. Ecosystem: what shipped this week

• DFlash2 (llama.cpp PR #27342, still in review): 2.26×–4.68× on real coding prompts, +2.7GB VRAM. N-max 5 beats the recommended 7; --spec-draft-p-min silently does nothing; stacking ngram-mod hurt (opposite of DFlash1 on 3.6).
• ninfer: Blackwell/5090-tuned engine; 120–160 t/s quants; 480 t/s at 4-way concurrency. Likely source of the unreproducible speed screenshots.
• AutoRound INT4 / AWQ-INT4 GGUFs for vLLM serving.
• KVarN 4/2-bit KV ported to vLLM 0.27.1 — 262k fits small cards, needle-test passes at 240k, ~20% slower decode.
• Uncensored/abliterated variants shipped fast: Huihui-ai ablit, an "Uncensored Aggressive" release bundling K_P quants + HauhauCS FastMTP (up to 3.02× TG claimed), and FP8 abliteration reporting refusal rates dropping to 0–6% — with the community counterpoint that the same tables show 30–50% caveat-rate degradation next to those numbers. Quality varies wildly; check benchmark deltas before switching.

What's coming

• 35B-A3B spotted in ms-swift commits (Aug 15). 16GB-card owners are hyped; early numbers suggest ~27–40 t/s on hardware where the dense 27B crawls.
• A new midsize open-weight model "next week (hopefully)" per Qwen's community manager — no early access this cycle. Speculation centers on ~80B with vision.
• The flagship Qwen3.8-2.4T-A95B got day-0 vLLM support with open weights announced at launch; it barely appears in this week's local-community threads beyond speed speculation (a 2.4T open-weight Call of Duty clone demo made rounds). Local discussion is overwhelmingly about the 27B.

Report template (steal this)

So your numbers mean something to the next reader:

Runtime/version:
Hardware:
Model file + quant:
KV cache:
Speculative (MTP/DFlash2/ngram):
Reasoning effort:
Sampling:
Context size:
Prefill tok/s:
Decode tok/s:
Task used:
Compared against:
Observed result:

Megathread compiled Aug 22, 2026 from r/LocalLLaMA and r/LocalLLM (Aug 15–22) plus public X benchmark threads. All performance figures belong to the hardware/runtime that produced them — the corpus contradicts itself on nearly every axis, and in most cases you can name the variable that explains the split. Prepped by Stealth ox alpha. Critic by Sol 5.6.

I pulled the Qwen 3.8 discussion from r/LocalLLM and r/LocalLLaMA from Saturday through today: 270 posts mentioning it, 208 with Qwen 3.8 in the title, then full comment reads on the higher-signal threads.

The early answer is not “local Opus.” It is more useful than that:

> Qwen 3.8 has real agentic-coding signal, but the difference between “wow” and “why is this unusable?” is often reasoning control, context, cache settings, quant, and runtime.

This is three days of community reporting, not a controlled model review. Still, a few patterns showed up too often to ignore.

The short version

Where the positive signal is real

The better reports were not “make a game in one prompt.” They gave the model a loop: write code, run it, inspect the result, fix it, repeat.

One hands-on comparison had Qwen 3.8 iterating a BASIC ray-tracer more successfully than Qwen 3.6 using the same Q8 quant. Another 16 GB setup reported a large coding run across more than one million processed tokens with only three prompts, using tests and linting as the correction loop. That is the useful story: a smaller local model that can stay in a real feedback loop without immediately falling apart.

But the pushback is healthy. Several commenters pointed out that viral arcade-game or clone demos can overlap with familiar training patterns. A model that makes a convincing game from a narrow prompt is not automatically good at a private repo, a messy codebase, or a long-horizon agent task.

The practical takeaway: if you are evaluating it for Hermes, use a task that has a verifier. A test suite, linter, structured output check, browser assertion, or human review loop will tell you far more than a one-shot demo.

The actual fight: it thinks too much

“Overthinking” was the dominant failure report.

One LocalLLM user said Qwen 3.8 was a step backward for documents and complex argument analysis because it kept following side paths after web retrieval. A legal-work reply in the same thread said their MCP-backed case workflow was outperforming prior local options. That is not a clean contradiction. It is a warning that the model's behavior is unusually sensitive to task design and configuration.

The recurring settings lesson is that reasoning effort and a thinking-token budget are not the same thing. Depending on llama.cpp version and UI, a selector can merely cap or cut off reasoning instead of changing the model's intended effort. That creates a lot of reports that sound like model behavior but are really template or runtime behavior.

A vLLM user on an RTX 6000 Pro reported basic responses taking one to five minutes even on low or medium reasoning. In a separate ongoing local evaluation, xhigh generated roughly 3.5 times as many tokens as medium, while medium was reportedly slightly better than Qwen 3.6 in score and used around 60% of the total generated tokens.

So “Qwen 3.8 overthinks” is not enough information to act on. The next useful report needs to say what actually controlled the reasoning.

Speed numbers are configuration cards, not rankings

The most useful throughput thread included the missing context most speed screenshots skip.

One RTX 3090 report using Q5_K_M went from about 75 tok/s around 1K context to about 43 tok/s around 90K after moving the model to SSD and enabling MTP, which is multi-token prediction/speculative decoding. Other reports in the same discussion ranged from dual 3060s and Strix Halo to RTX Pro 6000, dual 7900 XTX, and 5090 setups.

A detailed RTX 5090 llama.cpp report hit about 128 tok/s with NVFP4 plus MTP, well below widely repeated 200 tok/s claims. Replies pointed to the inference engine, Windows versus Linux, context reservation, and MTP acceptance rate as likely reasons. At 262K context, running out of VRAM for the KV cache, the model's context-memory cache, can turn a fast setup into a slow one.

The AMD report was the same lesson in a different suit: a 7900 XTX user saw roughly 25 to 35 tok/s around 80K context and near 20 tok/s around 150K to 180K. The likely culprit was KV cache overflow into system RAM, plus lower MTP-draft acceptance than Qwen 3.6 in one same-hardware comparison.

This is why a naked “I get 200 tok/s” post is not a benchmark. It is a postcard.

16 GB is possible. It is not magic.

The 16 GB community is getting real results, but the compromises are visible.

One RTX 5060 Ti guide ran a Q3 model with quantized KV cache at roughly 73K context for agentic coding. Another guide reported Q4_K_M around 130K context and about 20 tok/s. A 16 GB RTX 5080 report claimed 85.5 tok/s with an IQ3_XXS build and MTP at about 124K context.

Those numbers are useful, but no one should read them as “Qwen 3.8 runs great on every 16 GB card.” The question is which compromise you are making:

• lower quant quality
• less context
• lower context-cache precision
• slower generation under real context load
• more dependence on a particular runtime or custom template

For Hermes-style work, I would rather have a stable Q4 or Q5 setup with a realistic context window and a verifier loop than chase a screenshot from a very aggressive Q3 configuration.

What I would test before switching a Hermes workflow

1. Run your actual task twice: once at medium reasoning and once at your current default.
2. Measure first-token latency and decode speed at both fresh context and a realistic saturated context.
3. Record exact model file, quant, context size, KV cache, MTP setting, template, runtime version, and OS.
4. Use a task with a check: tests, a linter, a schema validator, a browser assertion, or manual acceptance criteria.
5. Compare total time-to-correct-result, not only tok/s.

That last line matters. A model that is 30% faster but spends twice as long reasoning, or needs three more repair turns, is not faster where it counts.

If you are posting numbers, use this

Runtime + exact version:
Hardware + OS:
Model file + quant:
Context size and current context depth:
KV cache type/quant:
MTP/speculative setting and acceptance rate, if available:
Reasoning effort and thinking budget/cap:
Temperature / template:
Prefill tok/s:
Decode tok/s at fresh and saturated context:
Concurrency:
Task and verification loop used:
Observed result, regression, or failure:

That would make the next week of Qwen 3.8 discussion dramatically more useful. The model looks worth serious local testing. The configuration is still doing half the talking.

Source threads

• Qwen 3.8 vs 3.6 ray-tracer agent loop
• Document-analysis criticism and legal-work counterexample
• Long review, plus the “do not call this local Opus yet” pushback
• 16 GB RTX 5060 Ti agentic-coding configuration
• Large Qwen 3.8 throughput field report thread
• 16 GB VRAM Q4_K_M guide
• Reasoning effort low/medium/xhigh comparison
• llama.cpp reasoning effort versus budget PSA
• vLLM endless-thinking report
• 7900 XTX context and KV-cache slowdown discussion
• RTX 5090 NVFP4 reproduction attempt
• 2× Intel Arc Pro B70 speculative-decoding report

What Hermes Agent Megathread topics do you want see roll out?

Put your comments below. If someone already mentioned it, upvote and comments under that comment.

https://preview.redd.it/88zlfqco1djh1.png?width=2507&format=png&auto=webp&s=f55187f75df75af4295f7be31c2653c7d71cec22

Just in time for the weekend. Post experiences here, thread will stay up over the weekend.

It’s Wednesday. Battle stations, open up.

Show us the real Hermes setup you’re running - desk, terminal, laptop, server rack, home lab, profile structure,voice setup, remote machines, browser automation, whatever makes it yours.

No need for a pristine lab or an expensive GPU rack. A MacBook, a Windows box in the corner, a VPS, or one terminal window held together by questionable YAML all qualify.

If you want a format: (optional)

─ text
Hardware:
OS:
Hermes version:
Favorite workflow:
Current weak spot:
Next upgrade:

Photos, screenshots, redacted config snippets, folder trees, and short walk through are all fair game.

One rule: redact API keys, tokens, passwords, private IP addresses, and anything else you wouldn’t want permanently attached to your Reddit account.

Extra Credit:

What’s one thing Hermes does for you now that you refuse to go back to doing manually?

Without further ado, Show the clean builds, the chaos builds, and the “it works, don’t touch it” builds.