Login

u/MikeNonect

r/LocalLLaMA r/StrixHalo
If you would have told me half a year ago that a local model running in my office would be able to one-shot a Super Mario clone, I would have called you nuts. Qwen3.8-27B is a different beast.
▲ 644 r/StrixHalo+1 crossposts

If you would have told me half a year ago that a local model running in my office would be able to one-shot a Super Mario clone, I would have called you nuts. Qwen3.8-27B is a different beast.

Running the Q8 GGUF on my Framework Desktop is not fast, but it's extremely smart for overnight batches and background jobs. Can't wait to play around with MTP and other quants.

Have any of you found ways to improve speed while keeping accuracy?

https://mikeveerman.github.io/qwen38-27b-mario

u/MikeNonect — 6 days ago
▲ 472 r/LocalLLaMA

Getting a feel for how fast X tokens/second really is.

I love following all your adventures with local LLM setups. Quality and size of the models are important, but so is performance. Numbers don't really convey the experienced speed well, however.

If someone claims they run Qwen 3.6-27B at 21 tokens/second, how fast is that? Is 10 tokens/second unusable? I find these numbers objective but meaningless.

I built a script that helps me get a subjective feel for these objective numbers.

It supports text, code and reasoning + code.

https://mikeveerman.github.io/tokenspeed/

reddit.com
u/MikeNonect — 3 months ago