
Is this real ? Qwen3.6:27b with 128k context fit in 24Gb VRAM ?
Hi,
I've been running Ollama on my Unraid server since the llama2 era. I use to be able to run qwen3.5 and then 3.6 27b with 32k context, barely but it was fitting. The other day I notice that the VRAM usage was WAY lower than I remembered.
So I kept increasing the context window, hitting 128k at 83% VRAM usage ! How ? Can I go further ?
I'm using the model in opencode right now and the token count goes up to 131072 before the awnser automatically stop (an issue I had before, but at 32k used) all this without using the system RAM and keeping a healthy 30 tok/s (sys ram would be 4 tok/s).
I'm using OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q4_0 as well.
Is this just an hallucination ?
edit : qwen3.6 Q4, or Q5 with 96K context works too!