Login

u/UnseemlyCorgi

r/LocalLLM r/oMLX
▲ 14 r/oMLX

Qwen3.6-27B-oQ8-mtp + Native MTP on M5 Max: stuck around 9–10 tok/s sustained - losing my mind

I've been hammering away at this issue for what feels like decades now. I'm using Jundot/Qwen3.6-27B-oQ8-mtp in oMLX with Pi as a coding harness and am only getting 9-10 ish t/s (generation, not prompt processing) to matter what I try...no matter what settings I fiddle with. 9-10 is the absolute max I'm getting. I'm hoping someone can suggest a fix as I've exhausted my non-expert knowledge and experience.

Hardware:

  • MacBook Pro M5 Max
  • 128GB RAM
  • 40-core GPU
  • oMLX running locally on LAN
  • Pi using the oMLX OpenAI-compatible endpoint

Model/settings:

  • Model: Jundot/Qwen3.6-27B-oQ8-mtp
  • Model Type Override: LLM
  • Native MTP: ON
  • TurboQuant KV: OFF
  • VLM MTP: OFF
  • DFlash: OFF
  • SpecPrefill: OFF
  • Thinking: OFF (for testing purposes)
  • Temp: 0.1
  • Top P: 0.95
  • Top K: 20
  • Context cap has mostly been 131072

An important details - oMLX originally auto-detected this model as VLM. In Pi, that caused the model to process one turn and then stop almost immediately. Forcing the model type to LLM fixed that behavior.

Now the issue is speed.

With Native MTP ON, a raw curl test outside Pi gives roughly:

  • prompt: 39 tokens
  • output: ~1400–1600 tokens
  • total time: ~149–161s
  • sustained speed: ~9.5–9.9 tok/s
  • MTP path is definitely active
  • MTP accept rate around 71–73%

Example log line:

MTP finish=stop tokens=1420 cycles=827 accept=591/827 (71.5%)
timing[backbone=132784.6ms mtp=6628.3ms sample=6732.0ms cache=79.6ms]
Chat completion: 1419 tokens in 148.90s (9.5 tok/s), prompt: 39

With Native MTP OFF, speed drops to roughly ~6 tok/s. So MTP is helping, but only by about 1.5–1.7x.

One interesting detail that might be relevant (honestly, I don't know at this stage of things). I had a period yesterday when I was getting 30 ish t/s for no reason at all (well, I'm sure there is a reason, I just have zero clue what it is). I went to bed happy thinking that my settings fiddling found the right combo, only to discover this morning that it was back to the glacial t/s rate.

I’m not looking to switch models right now. The goal is to get this exact MTP model working as fast as possible for Pi/coding-agent use and stop banging my head against the wall in frustration.

any help or suggestions would be appreciated beyond belief.

reddit.com
u/UnseemlyCorgi — 7 days ago
▲ 1 r/LocalLLM

Anyone else seeing "cache reuse is not supported" with Qwen 3.6 35B A3B in LM Studio?

I'm hoping someone here can help me sanity-check this because I'm at the point where I genuinely don't know where the problem lives anymore - and I'm completely at the end of limited knowledge and experience.

I'm running:

  • MacBook Pro M5 Max (128 GB RAM)
  • LM Studio 0.4.15+2
  • Qwen 3.6 35B A3B (Q8)
  • OpenChamber on top of OpenCode
  • Local LM Studio endpoint

The frustrating part of my situation is that the model itself seems to run fine.

Response quality is excellent.

Generation speed seems reasonable.

Memory usage is fine.

The problem that is driving me absolutely nuts is that once I start doing real work in OpenChamber (coding tasks, GitLab MCP, project skills, planning mode, etc.) everything becomes painfully slow - INSANELY slow. Yesterday, I ran a test on a single flat html file (with css and some light JS) guided by a plan mode plan. It took 8 hours to do 15 or so not very complicated edits.

Digging through the LM Studio logs, I repeatedly see messages like:

cache reuse is not supported - ignoring n_cache_reuse = 256

forcing full prompt re-processing due to lack of cache data
(likely due to SWA or hybrid/recurrent memory)

From what I can tell, it looks like LM Studio is repeatedly reprocessing prompts instead of reusing cached context.

I've spent the last couple days trying to track this down and quite frankly am losing my mind.

Things I've tried:

Specifically, I modified the template and added:

{%- set preserve_thinking = true %}

The preserve_thinking memory test talked about in a few reddit posts and github issues (below) works correctly, so that part appears functional.

Relevant discussions I've found:

LM Studio issue:
https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1563

Qwen cache-reuse discussion:
https://github.com/QwenLM/Qwen3/issues/1826

Recent Reddit discussion about preserve_thinking:
https://www.reddit.com/r/LocalLLaMA/comments/1sne4gh/psa_qwen36_ships_with_preserve_thinking_make_sure/

Reddit discussion on cache reuse / prompt drift:
https://www.reddit.com/r/LocalLLaMA/comments/1sg076h/i_tracked_a_major_cache_reuse_issue_down_to_qwen/

What's confusing me is that simple LM Studio chats seem fine, but once OpenChamber/OpenCode starts injecting larger prompts (system prompts, skills, MCP definitions, tool schemas, GitLab tools, etc.) performance falls off a cliff and the logs start complaining about cache reuse.

So I'm trying to figure out:

  • Is this a known Qwen 3.6 A3B issue?
  • Is this an LM Studio issue?
  • Is this a llama.cpp limitation with A3B/SWA/hybrid memory models?
  • Is OpenChamber somehow causing cache invalidation?
  • Or is this just expected behavior for this model architecture?

Seriously, I'm at my whit's end and have exhausted everything I can think of. Any ideas or guidance would be enormously appreciated.

reddit.com
u/UnseemlyCorgi — 24 days ago