TL;DR: I challenged myself to ship a real, working product using only a local model — no cloud, no shortcuts. Result: a Chrome extension — select text in any editable field, a floating action bar appears, pick an action (Fix / Improve / Translate), and the text is replaced in place by a local Ollama model. Each action has its own prompt, model and icon, and you can add your own presets. Private, fast, offline. 10-sec demo below.

Why I actually wanted this: I personally write most of my emails in a rush — typos, half-finished sentences, whatever comes out first. If you care at all about how the email you send looks, this genuinely helps: select, click Fix, done, still in your own words. Same for a Reddit post before you hit submit. It's not a novel idea — I already have an enhancer feature built into CO_DE (my main project). But I wanted to rebuild it from zero as a standalone Chrome extension that works everywhere in the browser. I knew exactly what I was aiming for going in.

Setup: unsloth/Qwen3.6-35B-A3B-MTP (Q4) on llama-server, Apple M5 Max 128 GB, MTP speculative decoding. Same server, same model for the whole build. ~5 hours across two sessions — struggled one evening, closed it the next.

Getting a local model to an actually-working, actually-useful result end-to-end is NOT trivial — it took real iteration to get here, and I'm not going to pretend it was a one-shot. If you try it yourself, expect to fight the model a bit. That's kind of the point of the challenge.

Asked the model itself what got fixed today — this is its own list, not mine:

Bugs fixed today:

Popup JS ID mismatch — HTML used lte-prefixed IDs, JS used non-prefixed ones. Every control returned null.

Global model selector — JS treated it as <select>, HTML had a <button>. Rewrote to use dropdown model selector.

Master toggle — JS used .checked (checkbox), HTML used custom button with on attribute. Fixed to toggle on attribute.

normalizeEndpoint was async without await — URL became "[object Promise]/api/chat". Made it synchronous.

window.__lteLoaded threw in cross-origin iframes — wrapped in IIFE + try-catch.

Content bar never appeared — bar was created only after chrome.storage.local.get() resolved (async). Fixed: bar created synchronously immediately on script load.

Focusin listener missed events — was using non-capturing mode. Changed to capturing.

Collapse/expand broken — toggleCollapse checked the wrong element and returned early. Rewrote.

Action cards were hardcoded open then broken — fixed default collapse state.

Sublabels overlapping content — <label> elements colliding with inputs/textarea. Changed to <div>.

Padding too tight — increased from 10px 12px to 14px 16px.

Still some polish left to do — but polishing with a local model isn't the best time/value spend past a certain point, so I called it here.

The extension: MV3, plain JavaScript, no build step, no external libraries, everything local (Ollama at localhost:11434). Works in Gmail, Reddit, anywhere text is editable. Popup config: endpoint, global model, per-preset prompt / color / model.

happy building local!

attached and the fixes, work from today

https://preview.redd.it/9csre9yv7seh1.png?width=2044&format=png&auto=webp&s=3d0f589e9cf91604f482dccea1092613310c60a1

--

EDIT — scope & method (quality, "just reading docs", Thinking, and the host app question):

Fair pushback on quality — it's a real axis. It's just not the one this test measures. This isolates harness overhead: same model, same weights, same prompt, same repo, Thinking on for all three (Qwen3.6-35B-A3B MTP on llama-server, local). The only variable is the harness. Quality depends on the model, which is identical here — it cancels out. A quality benchmark is a separate test, and I didn't claim otherwise.

"It's just a document-reading test": the task is a Next.js entry-point audit — pages, layouts, API routes, server actions, middleware, dead code + cleanup plan. It's what you do before touching an unfamiliar codebase. Full prompt on screen at 0:16, 3:16 and 6:17 (video: https://youtu.be/K85dmuet6xM)

"Isn't the extra context just Thinking?" Thinking is on for all three, same setting — constant, so it can't explain the gap. opencode's extra tokens are the harness re-sending the system prompt + full tool schema every turn (llama-server log: ~105–109k context, checkpoints 12→15 of 32; far less for the others). Throughput is identical for all three: ~46 t/s, ~75% MTP draft acceptance, prefix cache reused.

"Just hit the endpoint directly — faster": exactly. No harness is the floor; every layer adds wall-clock. This just ranks three harnesses on that axis.

"Is it your app skewing it?" I ran opencode twice. In-app it took 7:57; standalone in a plain macOS Terminal (no app) it did better — 4:29 — but was still the slowest of the three. And the ordering holds with a different model: same audit with GPT-5.5, Pi finished in 5:09, opencode in 6:45. Two models, two environments, same ordering — it's the agent loop. Screenshot of the run is in my reply to u/Longjumping-Music638.

Next up: a head-to-head coding test — same three harnesses, same local model, building the same mini-app from the same spec — for the quality side. Speed measured here, quality measured there. Will post the results.

--

Hardware: Apple M5 Max, 128 GB. Model / server (shared by all three): unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL on llama-server — -ngl 99 -c 262144 -fa on -np 1, MTP speculative decoding.

Results (wall-clock, same audit): CO_DE (in-app chat, Coder mode) ~2:00 · Qwen native CLI (Qwen Code) ~2:30 · opencode CLI 7:57 in-app / 4:29 standalone (best run).

Why opencode is the outlier (from the llama-server log):

• Throughput is identical for all three: ~46 t/s generation, ~350 t/s prefill, MTP draft acceptance ~75%. Prefix cache is reused (sim_best 0.996), so no from-scratch reprocessing.
• opencode's context ballooned to ~105–109k tokens (checkpoints 12→15 of 32) vs. far less for the others.
• Its agent loop is heavier: large system prompt + full tool schema re-sent each turn + more exploratory tool round-trips, each appending to a growing context.

Takeaway: For local agentic work, wall-clock is dominated by how much context the harness pushes and how many tool turns it takes — not raw t/s. The same model can be 2× apart depending on the harness. opencode is heavier by design; leaner harnesses (or trimming tools / AGENTS.md) close the gap. In my opinion opencode is not the best choice for coding with local models — I say this because I see many using it that way.

happy running local models!

Same repo, same starting point, paralel run, and the exact same GPT prompted all three workers:

Pi + Ornith 35B 1M MTP did the task in 4:24

Pi + Qwen3.6 35B A3B MTP did it in around 3:59

and added Claude Sonnet 5 just to have a cloud reference work, 2:36.

All three completed the task.

GTP conclude: Final quality verdict: good enough to trust the overlapping findings. Both PI workers independently agreed on the most important issues"

My conclusions on testing them,

• Ornith 35B 1M MTP: Very fast, but unpredictable. It sometimes cuts tasks short or moves so quickly that it is difficult to verify whether every requirement was handled. It does not consistently respect the workflow and tool rules.
• Qwen3.6 35B A3B MTP: More disciplined, consistent, and predictable. It follows instructions and tool contracts much more reliably. Although generally a little slower, it finished earlier in this run. This is the local model I would currently trust as an orchestrator.

These are observation from a day parallel run, not a definitive benchmark. For agentic work, raw speed matters less than predictable behavior and reliable completion.

A flavour of the speed of the locals running in Pi during test here:

https://youtu.be/6J6V2kMohwg

(in some tests today Ornith 35B was twice as fast as Qwen3.6 ont he same audit promt, 2:02 vs 5:46, i forgot the tests where cli and orchestrator same model with np -2 )

see you around! happy codding >_!

Straight, the conclusion of the run on apple m5 max: ik_llama.cpp won this CPU/ARM comparison clearly. It was 20.8% faster overall, with its largest advantage in text generation. Vision and post-vision generation also remained faster.

Result

Case	ik_llama.cpp	llama.cpp	Lead
Overall	65.71 tok/s	54.27 tok/s	+20.8%
Text	78.67	59.86	+31.4%
Vision	59.84	51.57	+16.0%
Post-vision	58.62	51.38	+14.1%

One particularly large difference appeared during vision prompt encoding: approximately 123.7 vs 21.6 tok/s, or around 5.7× faster for ik_llama.cpp.

Hardware & software used

• Apple M5 Max, 128 GB unified memory
• The benchmark was run and monitored through CO_DE, my free local-first CLI and agent workspace. It handled the commit matrix, interrupted-run recovery, winning-build installation, and reproducible report packaging.
• Qwen3.6-35B-A3B-MTP UD-Q4_K_XL
• Matching mmproj-BF16.gguf
• CPU-only: ARM NEON + Apple Accelerate, Metal disabled
• Three rounds per case: text, vision, and text after vision
• Median generation speed per case; aggregate is the mean of those medians

CUDA users: you have a friend in me. I also prepared a CUDA/NVIDIA version of the same reproducible test harness. It is not part of this Apple Silicon result, but anyone interested can ask in the comments or message me and I’ll share the download link.

Links sources and the test movie:

• ik_llama.cpp commit 1fddd12ba861
• llama.cpp commit 0dc74e332ede
• https://github.com/gelubodrug/co_de
• https://youtu.be/ACZg9tgPfqA

I WIll come back with test on other models i choose, or a specific one if you point it to me in comments and i can run it on my hardware, non CUDA.

Happy >_ codding!

Conclusion: No. In this test, using vision did not disable speculative drafting from the MTP head, either during the vision turn or for the rest of the session. I've seen worries in reddit post about this aspect. maybe was real in the past. now it s gone.

Tested with llama.cpp b9620 (57fe1f07c), an M5 Max, and unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL.

2-minute video: https://www.youtube.com/watch?v=lcXdfkXdLE0

Launch

llama-server -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
  --mmproj .../mmproj-BF16.gguf \
  -ngl 99 -c 262144 -fa on -np 1 \
  --spec-type draft-mtp --spec-draft-n-max 2 --port 8081

Both subsystems initialize in the same process. llama.cpp reports separate memory budgets for the projector and MTP context:

load_model: [mtmd] estimated worst-case memory usage of mmproj is 1134.00 MiB
load_model: [spec] estimated memory usage of MTP context is 826.70 MiB
common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
common_speculative_impl_draft_mtp: n_max=2, n_min=0, p_min=0.00, n_embd=2048, backend_sampling=1
load_model: speculative decoding context initialized
load_model: loaded multimodal model, '.../mmproj-BF16.gguf'

Text turns

Five text turns, with a prompt reaching roughly 29k tokens. Drafting remained active throughout:

task	draft acceptance	eval tok/s
1	0.862 (100/116)	95.29
78	0.955 (168/176)	99.76
171	0.828 (53/64)	89.36
209	0.739 (525/710)	79.48
570	0.865 (83/96)	88.65

Vision turn

Same server and session, now with a turn that actually encoded an image:

slot process_mtmd: id 0 | task 627 | encoding mtmd batch from idx = 82, n_chunks = 1
slot print_timing: task 627 | eval time = 6801.48 ms / 712 tokens
slot print_timing: task 627 | draft acceptance = 0.61950 (394 accepted / 636 generated)

That is 104.68 tok/s, the fastest eval measured in this session. While an image was in context, the MTP head generated 636 draft tokens and 394 were accepted.

Two messages appeared during the run:

• find_slot: non-consecutive token position 82 after 81 ... during image encoding. Generation and drafting continued.
• Qwen-VL models require at minimum 1024 image tokens ... try adding --image-min-tokens 1024. This is worth enabling for OCR or grounding, but was not required for a plain image description.

Results

Drafting survived the image. Acceptance fell to 0.62 compared with roughly 0.83-0.95 on the earlier text turns, but drafting was still active. The head was simply less confident about tokens following an image.

Drafting survived the session. Task 949, a text turn after the image, still drafted at 0.680 acceptance (151/222).

Final counters across seven turns:

generated drafts = 1010
accepted drafts  = 818
generated tokens = 2020
accepted tokens  = 1474

That is roughly 73% accepted overall.

No throughput penalty was observed in this run. The vision turn was actually the fastest measured, although this single test does not establish a universal performance result.

So, on llama.cpp b9620, --mmproj and --spec-type draft-mtp coexist. If you previously saw MTP drafting stop when vision was enabled, it may be worth retesting on a current build.

Happy codding!

Benchmark complete: 153/153 successful samples across 17 commits, with no failed requests.

Thanks to u/SM8085 for suggesting a commit-by-commit benchmark and pointing me toward the relevant regression range. That idea triggered this full 153-run test.

In the first comment, find the link to the testing setup on Git to make your own tests.

Conclusions

Best: 57fe1f07c3b6 (b9620)
Published: June 13, 2026, 11:51 CEST
Score: 95.95 tok/s
Text: 89.82, Vision: 98.80, Post-vision: 99.24 tok/s

Slowest: b11f7c16bc1c
Published: June 26, 2026, 08:43 CEST
Score: 75.35 tok/s
Text: 72.75, Vision: 82.61, Post-vision: 70.70 tok/s
No official build tag.

b9620 was 27.34% faster than the last-place commit.

The setup

Apple M5 Max with 128 GB unified memory. I launched, paused/resumed, monitored, and collected the benchmark reports through CO_DE, my local agent ide, to run and monitor the commit matrix, resume interrupted runs, install the winning build, and package the reproducible reports.

llama-server \
  -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
  --mmproj /path/to/mmproj-BF16.gguf \
  -ngl 99 \
  -c 262144 \
  -fa on \
  -np 1 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --port 8081

After confirming that --mmproj and --spec-type draft-mtp can remain active in the same vision session, I wanted to answer the next question: which llama.cpp commit actually performs best?

A performance regression has been reported across recent builds, but comparing numbers from different prompts, machines, context lengths, and sampling settings does not isolate the commit. I therefore ran the relevant history headlessly on one machine with one model, one projector, one image, and one fixed prompt. The tested llama.cpp range was 6ee0f657 through b9935.

The anchor set includes the first MTP + image-support commit (6ee0f657), the community-reported fast commit (e9fb3b3f), the reported regression boundary (b9222/b9235), my previously tested build (b9620), and b9935. A coarse first-parent scan is followed by an automatic refinement around the two fastest commits. Commit selection comes directly from Git history; it is not hand-picked after seeing results.

Reproducible protocol

Every commit is built in Release mode with Metal enabled, then tested in its own fresh llama-server process using the setup above.

Each build receives the same roughly 29K-token code prompt, image, deterministic sampling (temperature=0, seed=42), and 256-token output cap. Three rotating rounds are run for each of these cases:

1. Text-only generation
2. A real image-encoding turn
3. Text generation after the image remains in the conversation

The runner records llama.cpp's own eval throughput, prompt throughput, MTP draft acceptance, accepted/generated draft counts, TTFT, wall time, failures, and raw server logs. Completed cases are checkpointed, so pausing and resuming does not silently repeat or discard them. Benchmark configurations are fingerprinted to prevent short smoke tests from contaminating the full run.

Overall ranking

Rank	Commit	Text tok/s	Vision tok/s	Post-vision tok/s	Vision draft acceptance
1	57fe1f07c3b6 (b9620)	89.82	98.80	99.24	0.7700
2	2f18fe13c5dd	89.99	100.56	96.31	0.7828
3	f2d1c2f3984c (b9935)	89.95	100.59	96.28	0.7828
17	b11f7c16bc1c	72.75	82.61	70.70	0.7828

The winner was not the fastest in every individual column. 6ee0f657 had the highest isolated vision median at 100.74 tok/s, while several top commits were effectively tied around 90 tok/s for text. b9620 won the aggregate largely because it retained 99.24 tok/s after the image turn. This is why ranking one cherry-picked request would have produced a different answer.

The b9222 and b9235 anchors both reproduced the slow region at 76.80 and 77.18 aggregate tok/s respectively. The later b9620 build recovered strongly. Draft acceptance alone does not explain the ordering: the winner's vision acceptance was 0.7700, slightly below several neighboring commits at 0.7828.

What this result does and does not mean

This identifies the fastest commit in this controlled Apple Silicon setup for this model and workload. It does not establish a universal winner for CUDA, other quantizations, different context lengths, or different MTP heads. The raw logs matter: a build should not be called faster merely because it accepted fewer drafts, produced a shorter answer, failed to encode the image, or changed the tested path.

The complete runner, commit manifest, CSV, Markdown summary, and raw per-case logs are included so the result can be reproduced or challenged. For this specific Apple Silicon workload, b9620 is the build I would reproduce first.

Happy codding! >_

I personally bet that it would die on context because of the expected long run turn.

Claude bet would die on other 2 points, more technical

BUT IT Didn't, big milestone for a solo dev testing it's app.

What broke was dumber, and then it got interesting.

The Setup: one repo, three branch workspaces, three CLI workers — Claude Opus 4.7

(medium), GPT-5.5 Codex (medium), and a local model running in Pi. Supervising them:

Qwen3.6-35B-A3B-MTP on llama.cpp b9620, on my own machine, 256k ctx, MTP speculative

decoding on.

Six steps it had to gate: plan, approve, build, report, verify, brief —

then reconcile all three branches onto main.

It ran 30 minutes.

My bet going in was context. Three workers, six steps each, every report landing back in

one window. I expected it to lose the thread.

It didn't. The 35B (3B active) picked its workers from the live catalog instead of

inventing names, spawned all three on isolated branches, gated every plan before any code

was written, tracked who was on which branch for half an hour, and never mixed them up.

What broke was this: none of the workers committed.

All three finished. All three reported done. Every branch was left dirty, so reconcile

couldn't run. I went and read my own source afterwards: the word "commit" appears nowhere

in the coordinator's instructions, and nowhere in the report contract the workers get.

Worse — Claude workers run markerless in my harness, deliberately given no contract at

all, because their transcript is read directly. So one of the three could never have been

told, even if the contract had said it.

Then the interesting part. The coordinator noticed, committed on each worker's behalf,

and merged by hand with git merge — because my own approval gate for the reconcile tool

only accepts a UI button, not an "ok" in chat. So it routed around the gate.

I checked: the gate blocks the reconcile tool and git push. It does not block git merge.

Same promotion into main, completely ungated, and none of the reconcile checks

(plan/build/verify/promote) ran. A gate on the tool, with the shell left open, is not a

gate — it just moves the work to the path where you have no telemetry.

Final state on main: 11 files changed, +193/-65.

Opus: styled UI output across 7 CLI modules

Pi (the local one): a new "hajimi go" subcommand

GPT-5.5: 7 real bug fixes across 6 files

If you build anything that delegates to coding agents: go check whether anything in your

stack tells the worker to commit before it reports done, and whether your gates sit on

tool names or on actual state. Mine failed both. I only found it because I let a run go

30 minutes instead of 30 seconds — a short demo passes clean.

The honest part***: it finished end to end because I was watching.*** When reconcile looked

wrong I opened the code, screenshotted it, and showed it to the coordinator — it read the

screenshot and corrected course. That is a human in the loop, not autonomy. In a headless

run nobody takes that screenshot.

Fixed since: a branch task now tells the worker to commit before reporting. The gate is

not fixed yet — blocking git merge just moves it to rebase, so it has to move onto state,

not command names. Next run I'm doing the same thing remote from Telegram, and I'll come

back with results.

So Happy codding!

My thoughts first:

My takeaway from this blind benchmark: Pi with the local unsloth/Qwen3.6-35B-A3B-MTP-GGUF model is surprisingly strong.

It rendered very fast, found the real issue, and produced a result that can stand next to much more expensive Gemini/GPT-style agents for this kind of repo audit. The quality was not just “good for a local model” — it was genuinely competitive.

Meanwhile, the same local model through OpenCode took roughly twice as long in this run and did not add anything meaningfully better than the faster Pi run. It was still useful, but Pi gave the better speed-to-quality ratio.

So my conclusion: for local coding audits, Pi + Qwen3.6 is not a toy setup. It is fast, cheap/private, and in this benchmark it held its own against premium-style agents. OpenCode with the same model worked, but Pi felt like the sharper runner.

The Test

Blind CLI benchmark — two turns, four CLI workers total.

The Orchestrator:

GPT-5.5 - Codex

Editable worker slots:

- CLI 1: Pi local unsloth/Qwen3.6-35B-A3B-MTP-GGUF:U

- CLI 2: OpenCode local unsloth/Qwen3.6-35B-A3B-MTP-GGUF:U

- CLI 3: Pi local Qwythos-9B-Claude-Mythos-5-1M-MTP-Q

- CLI 4: Gemini 3.5 Flash Medium

Pause:

- After Turn 1, STOP so the user can change the local model.

Goal:

Test multiple CLI agents as independently as possible on the current active repository. Do not contaminate later agents with earlier findings.

Rules:

- Do not edit code.

- Each worker must receive the SAME blind audit prompt.

- Do NOT tell any worker what another worker found.

- Do NOT include prior reports, hints, conclusions, or suspected bugs in any worker prompt.

- The coordinator collects reports privately and only compares them at the end.

- Use the current active repo/branch/desk context; do not hardcode repo paths.

- Focus area: one subtle bug around session/history behavior, especially --temporary, --stream, and saved/resumed sessions.

Blind worker prompt:

Audit the current active repository. Do not edit code.

Find one subtle bug around session/history behavior, focused on:

- --temporary

- --stream

- saved/resumed sessions

Do not assume there is a bug. Inspect the code independently and report only your strongest finding:

1. exact bug,

2. user-visible trigger,

3. code path causing it,

4. affected behavior,

5. minimal conceptual fix.

Do not mention or use any prior agent findings.

Turn 1:

1. Launch CLI 1 with the blind worker prompt.

2. Wait for CLI 1 report.

3. Launch CLI 2 with the exact same blind worker prompt.

4. Wait for CLI 2 report.

5. STOP. Do not inspect code yourself yet. Do not summarize final results yet. Wait for the user to change the model and explicitly start Turn 2.

Turn 2:

1. Before launching anything, show the plan:

   - CLI 3 and CLI 4 will be launched,

   - both are brand-new workers,

   - Turn 1 workers will not be reused,

   - both receive the same blind prompt,

   - no prior findings will be shared.

2. After approval, launch CLI 3 with the same blind prompt.

3. Wait for CLI 3 report.

4. Launch CLI 4 with the same blind prompt.

5. Wait for CLI 4 report.

6. Only after all four reports are collected, inspect the code yourself.

7. Final report:

   - correct bug,

   - which of the four reports were right vs wrong,

   - hallucinations or missed issues,

   - clarity ranking of the workers,

   - best overall model/CLI for this task.

Important:

This is NOT cascade verification. It is a blind comparison. The agents must not see each other’s outputs.

Happy codding!

Qwen3.6-35B-A3B-MTP blew me away on speed (MTP spec decode, ~130-150 t/s local). Then I attached a screenshot, asked it to read it — nothing. Looked text-only, and llama-server won't take images without a projector.

Turns out the fix is right there: the unsloth GGUF repo ships an mmproj-BF16.gguf next to the weights. Point --mmproj at it and the SAME fast MTP model reads images natively. That's the whole trick (pic 1, highlighted field):

--mmproj ~/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-MTP-GGUF/snapshots/<hash>/mmproj-BF16.gguf

Full launch:

llama-server -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \

--mmproj <that mmproj-BF16.gguf> \

-ngl 99 -c 262144 -fa on -np 1 --spec-type draft-mtp --spec-draft-n-max 2 --port 8081

Test (pic 2): same screenshot, "what do you see?", side by side —

- local Qwen3.6-35B + mmproj: read it correctly in ~30s (the image came out to ~31k tokens after the vision encoder; prompt processing ~1200 t/s, decode ~90 t/s)

- Claude Sonnet: same image, ~14s

So Claude is ~2x faster (cloud), but the local one is genuinely reading images now — no cloud, no tokens, on the fast MTP model. For a 35B at MTP speed, 30s to read a dense full-screen screenshot is fine by me.

The reason I almost missed it: you see a text GGUF and assume text-only. The mmproj was sitting in the repo the whole time.

Happy codding! _>

https://reddit.com/link/1uw2dmw/video/5f4arl7jk5dh1/player

Did another run (pi obliterated opencode...honest)— this time zero cloud, everything on one local model. Screens: setup, running, verdict.

One model, Qwen3.6-35B-A3B (unsloth UD-Q4_K_XL) on llama.cpp with MTP. It wears three hats in the same run:

1. the orchestrator — plans the task, spawns the workers, reads the reports, writes the head-to-head

2. worker A — a local opencode CLI

3. worker B — a local pi CLI (same model)

Cascade this time (one after the other, not parallel), so -np 1 is enough.

The prompt "lets make a quick test, prompt an opencode cli and a pi cli, ONE AFTER ONE, not both at the same time — cascade — to do a quick audit on the entry points in this repo, brief head-to-head report after both finished"

Results (last pic): both caught the critical stuff on their own — a documented mcp_server.py that doesn't exist, dead Pydantic models, no auth pre-checks. Where they split:

- pi was ~2x faster (~35s vs ~80s) and caught more operational stuff (silently-ignored globs, tracebacks masked by click.Abort(), stress-test stubs).

- opencode was slower but caught more code-level bugs (Windows os.uname breakage, a bare except, a stale package name, a stray Romanian comment).

The orchestrator read both and wrote the verdict: quick audit -> pi's speed wins; deep inspection -> opencode's findings were richer.

Numbers:

- ~78-81 t/s decode, MTP draft acceptance ~70%, 256K ctx, np 1.- wall-clock, one run each (not a controlled bench): opencode ~80s, pi ~35s.

no cloud, no tokens. One local model did the coordinating, both audits, and the judging. Whether that's genuinely useful or just a neat way to keep a GPU busy — curious what people think

Happy codding!

Everyone benchmarks t/s. I wanted to see a local model do real work AND coordinate it at the same time — so here's a run. Screens: setup, running, results.

One local model, Qwen3.6-35B-A3B (unsloth UD-Q4_K_XL) on llama.cpp with MTP spec decode. I launched llama-server with -np 2 and gave it two jobs at once:

1. be the orchestrator — plan the task, spawn the workers, read their reports, write the head-to-head

2. BE one of the workers — a local opencode CLI on the same server

So the local model runs the meeting AND does half the coding. Both jobs sit on the same GPU; the log shows both slots decoding during the run. The other worker is a paid cloud CLI (Claude Sonnet). Same prompt, in parallel.

The prompt (bit garbage but intended so): "lets make a quick test, prompt an opencode cli + a sonnet cli to do a quick audit on the entry points in this repo, brief head-to-head report when done"

Results (last pic): both landed the same top findings on their own — a documented mcp_server.py and generate.py that don't exist, a gem console-script name collision, a dead --mcp flag. Sonnet was a bit more thorough on the missing files, opencode more conservative on confidence. The orchestrator diffed both into the head-to-head you see.

Numbers:

- single stream, np 1: ~150 t/s, draft 20/20.

- 2-slot run: ~40-55 t/s per slot, but each slot is on ~32-34k of context, so it's slower for two reasons at once (concurrency AND big prompts) — not comparable to the tiny bench. MTP draft accept still ~72-74% under 2-way load.

- same task, wall-clock: np 1 ~1m40s -> np 2 1m14s.

- -np 2 splits the window: with -c 262144 each worker gets 256K/2 = 128K. Fine for a scoped audit.

Not claiming combined t/s went up — tiny prompt vs huge prompts, can't compare. The point isn't t/s; it's that a local model can run the whole thing AND pull its weight as a worker on a real task.

Cost, for what it's worth: coordinator is local and one of the two workers is local, so only the cloud worker is on the meter — a two-model head-to-head that costs one model's tokens.

If somenone want to see more tests combinations premium cli s vs Qwen3.6-35B-A3B MTP, let me know.

Happy building!