r/AgentZero

Qwen dropped 3.8 27B yesterday, so I rented an NVIDIA H200 to see how far the 27B could go on a real coding task in a powerful GPU like that.

Setup

- Qwen3.8-27B, served with vLLM

- 129GB VRAM allocated (weights + KV cache preallocation)

- 64k context window

The target codebase is Agent Zero and its sidebar. But this post is about the model, the same run should transfer to any decently structured codebase

I gave it one screenshot of our sidebar and a short prompt: turn the chat list into a WhatsApp-style list of agent profiles, avatar circle on the left, one persistent chat per profile. We don't have a plan mode in Agent Zero, but Qwen started planning first, and then executed it across 1h49m of continuous work (only one nudge because it couldn't get past the login screen with the masked password, fair)

The interesting part: 64k is nowhere near enough to hold the codebase, so it kept re-reading files right before editing them to make sure they were actually in the context window. It never edited from a stale memory of a file. That level of context discipline is something I mostly associate with much larger models and it was impressive

Anyone else tried it? Curious how it compares for you against much bigger models or even hosted frontier models on multi-hour tasks. This may be one of the best local AI models out there.

I have multiple Agent Zero instances, and one of them is still running v0.98 (yeah, I know, it’s old as hell 😂).

But honestly, for what I use it for, I find it much better than the newer versions.

I tested the new DeepSeek Flash and Pro on both the latest Agent Zero version and v0.98, using the exact same model, prompt and task.

On v0.98, DeepSeek is fucking smart. It does the task extremely well and sometimes I’m genuinely surprised by what it manages to do.

On the latest Agent Zero, the result is... almost acceptable.

The difference is honestly night and day.

And it’s not just one test. I’ve tried multiple tasks and I keep getting the same result. What’s even more interesting is that the older v0.98 instance also uses fewer tokens for the same task.

In one case, the difference was around 1.5 million tokens.

So what exactly changed between these Agent Zero versions that can cause such a massive difference?

Same model, same architecture, same prompt, same task — but completely different results.

Is there something in the newer Agent Zero versions that changes how DeepSeek reasons, manages context, prompts itself, or handles the agent loop?

I’d really like to understand what’s going on here.

Qwen3.8-27B ran a 2 hour autonomous coding session and cloned Grok Bot. Its context management surprised me

Why is DeepSeek much smarter on the older Agent Zero version?