u/Striking_Difficulty1

I've been using QuantaMind (free desktop app) to properly benchmark local models before trusting them in any agent workflow. Ran the built-in Coding eval suite today on two Ollama models and the results were more dramatic than I expected.

**What I ran:**
- Eval: Built-in Coding collection (Easy tier, 5 agentic tasks)
- K = 5 (each task run 5 times to test consistency, not just one lucky pass)
- Backend: Ollama on my local machine (64 GB RAM)
- Decoys: off

**Results:**

Model	Pass^(K)	Avg Steps	Effort	Top Error
gemma4:e4b (Q4_K_M)	3/5	1.8	257 tok	FAKE DONE
qwen3.6:35b (Q4_K_M)	5/5	1.8	447 tok	NONE

https://preview.redd.it/rbxpny8vzu8h1.png?width=3456&format=png&auto=webp&s=6e4eb6c43d15f2bd34f654fe29e2bfe40b68a91c

https://preview.redd.it/9fu8fx8vzu8h1.png?width=3456&format=png&auto=webp&s=01af6d29cfceffe219edc6add36e555cd27a548d

**The thing that stood out:** both models took the same average number of steps (1.8). The difference wasn't how many actions they took — it was whether those actions were actually correct.

The "FAKE DONE" error on gemma is particularly bad for agentic use. It means the model claimed to complete the task before it actually ran all the required steps. That's a silent failure — in a real pipeline it would just look like a successful run.

qwen3.6:35b passed everything cleanly, but used significantly more tokens (~74% more). So there's a real cost trade-off depending on your setup.

**Tool I used:** quantamind.co — free macOS app, runs fully local, no data leaves your machine. Has a built-in agentic sandbox with anti-cheat rules (it specifically catches the fake-done pattern). Also has a context cliff probe and readiness verdicts if you want to go deeper.

Curious if others have seen the fake-done failure on smaller models, or if there are other 4B-class models worth testing against qwen3.6:35b on coding tasks.

I have recently bought a macbook pro m5 with the expectation of running local models locally and save money paying big tech.
But the reality is different local llm is still in the beginning stage and we can't get the same result as cloud models.
Still I am able to get some decent responses and functionalities from few of the models whihc I used with claude orchestration and able to do quite a work which saves my coding time.
I used the benchmarking tool which tests model which I downloaded locally and tells me is the model capable of doing the jobs accurate enough to do easy , medium , hard iterations of jobs withing minutes and saved my time on selecting the best model which I can use.

If you guys have found any model which is really good
Please feel free to share and discuss in this thread.
If you guys want to know which model currently im using comment "model" ill pin it.

And the model name with the quantization and how to use it to get better result in the upcoming post.

I've been using QuantaMind (free desktop app) to properly benchmark local models before trusting them in any agent workflow. Ran the built-in Coding eval suite today on two Ollama models and the results were more dramatic than I expected.

**What I ran:**
- Eval: Built-in Coding collection (Easy tier, 5 agentic tasks)
- K = 5 (each task run 5 times to test consistency, not just one lucky pass)
- Backend: Ollama on my local machine (64 GB RAM)
- Decoys: off

**Results:**

Model	Pass^(K)	Avg Steps	Effort	Top Error
gemma4:e4b (Q4_K_M)	3/5	1.8	257 tok	FAKE DONE
qwen3.6:35b (Q4_K_M)	5/5	1.8	447 tok	NONE

https://preview.redd.it/rbxpny8vzu8h1.png?width=3456&format=png&auto=webp&s=6e4eb6c43d15f2bd34f654fe29e2bfe40b68a91c

https://preview.redd.it/9fu8fx8vzu8h1.png?width=3456&format=png&auto=webp&s=01af6d29cfceffe219edc6add36e555cd27a548d

**The thing that stood out:** both models took the same average number of steps (1.8). The difference wasn't how many actions they took — it was whether those actions were actually correct.

The "FAKE DONE" error on gemma is particularly bad for agentic use. It means the model claimed to complete the task before it actually ran all the required steps. That's a silent failure — in a real pipeline it would just look like a successful run.

qwen3.6:35b passed everything cleanly, but used significantly more tokens (~74% more). So there's a real cost trade-off depending on your setup.

**Tool I used:** quantamind.co — free macOS app, runs fully local, no data leaves your machine. Has a built-in agentic sandbox with anti-cheat rules (it specifically catches the fake-done pattern). Also has a context cliff probe and readiness verdicts if you want to go deeper.

Curious if others have seen the fake-done failure on smaller models, or if there are other 4B-class models worth testing against qwen3.6:35b on coding tasks.