Unsloth updated qwen3.8 27b but they didn't include a change log.

To me it looks like the UD quants were updated and mtp has been stripped out to be seperate. I am not sure what else changed.

I really appreciate unsloths work but I don't enjoy not knowing if I should download a 30gb file again.

Edit: removed an incorrect statement. Mods feel free to remove this post when Unsloths official announcement is released.

I've been messing around with Qwen3.8 27B locally and I'm wondering if I'm getting the performance I should be getting or if my setup/config could be improved.

PC:

Ryzen 7 7800X3D

RTX 5070 Ti 16GB

RTX 3060 12GB

32GB DDR5-6000 CL30

Windows 11

llama.cpp / llama-server latest build

I'm currently running the Qwen3.8-27B UD Q4_K_XL GGUF with both GPUs using tensor split.

My current config:

llama-server.exe ^

-m "Qwen3.8-27B-UD-Q4_K_XL.gguf" ^

--alias "Qwen3.8-27B-UD-Q4" ^

--host 0.0.0.0 ^

--port 8035 ^

--n-gpu-layers 99 ^

--split-mode tensor ^

--tensor-split 60,40 ^

--main-gpu 0 ^

--parallel 1 ^

--flash-attn on ^

--cache-type-k q8_0 ^

--cache-type-v q8_0 ^

--ctx-size 131072 ^

--batch-size 2048 ^

--ubatch-size 512 ^

--threads 8 ^

--threads-batch 8 ^

--presence-penalty 0.0 ^

--repeat-penalty 1.0 ^

--temp 1.0 ^

--top-p 0.95 ^

--top-k 20 ^

--min-p 0.0 ^

--jinja ^

--reasoning-format auto ^

--no-mmproj-offload ^

--spec-type draft-mtp ^

--spec-draft-n-max 3 ^

--mmproj "mmproj-BF16.gguf" ^

--metrics

With MTP I'm getting around 40–46 tok/s depending on the run. I've seen around 41 tok/s pretty consistently, with n-max 3 seeming to be a little better than 2 for me.

Both GPUs are basically maxed during generation.

I'm mainly wondering:

Is ~40–46 tok/s reasonable for this hardware/config?

Is there anything obviously wrong or inefficient in my setup?

Would a different quant be a better choice for these GPUs? I've been looking at Ridge 3.7bpw, Q4/Q5 UD quants, etc.

Would it make more sense to use a smaller quant that could fit mostly/all on the 5070 Ti instead of tensor-splitting across both GPUs?

Is there anything I should change with the KV cache, batch/ubatch, tensor split, MTP settings, etc. to get better generation speed?

I'm mostly interested in coding/agent use through OpenCode, so I'd rather have a good balance of quality and speed than just chase the highest possible tok/s.

If anyone is running Qwen3.8 27B on a similar setup, I'd be interested to know what quant/config you're using and what kind of speeds you're getting.

There is no DFlash2 in Unsloth Desktop app. Where do you plan to add it?

As title.

Using vLLM with this params:

unsloth/Qwen3.8-27B-NVFP4

--dtype auto

--safetensors-load-strategy=prefetch

--tensor-parallel-size 1

--attention-backend flashinfer

--performance-mode interactivity

--language-model-only

--skip-mm-profiling

--kv-cache-dtype fp8_e4m3

--gpu-memory-utilization 0.94

--cpu-offload-gb 0

--max-model-len 128000

--max-num-seqs 1

--max-num-batched-tokens 6144

--enable-chunked-prefill

--enable-prefix-caching

--no-disable-hybrid-kv-cache-manager

--reasoning-parser qwen3

--default-chat-template-kwargs '{"enable_thinking": false}'

--enable-auto-tool-choice

--tool-call-parser qwen3_coder

--quantization compressed-tensors

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

I can't go past 128K context with 32GB of VRAM.

Is there a way to achieve 200K context?

I can't extend --gpu-memory-utilization 0.94

cause I need that vram for the destkop.

I basically moved from VS Code + ZOO-CODE and also LM Studio BIONIC and trying out Unsloth Desktop but I guess it's still very young with lots of issues.
BTW - I'm using the most up to date unsloth Desktop app for Windows 11

---

In LM Studio BIONIC and any other Agentic / Harness app:
While the model is "cooking" I can go to read other chats freely but when I did it in Unsloth Studio I found out in the hard way... while Qwen 3.8 27B was cooking which takes a LOT OF TIME in my RTX 5090 32GB VRAM...

Yes, I'm using MTP enabled ✅ but it doesn't change much at least from what I test, I'm still on around 30-37 tps (compare to other modes I fly with 100-220 tps but not this one).

---

After about 20+ minutes of processing and working I wanted to read a different chat and...
the moment I moved to the other CHAT TAB and came back... I found out, IT STOPPED!
it did not continue processing as expected compare to any other app I used for the same purpose.

For now I guess I'll have to go back to LM Studio BIONIC or VS Code + ZOO-CODE because this is critical in my opinion as a user at least.

But I fully understand that the app is young and I must give the wonderful unsloth dev team some time to improve it, I'm not giving up on the unsloth team I appreciate and LOVE what you guys are doing!

I REALLY want to get into Qwen 3.8 27B and test so many things and it's not helping with the current way it CUT the process while browsing other chats, hopefully other browsing/navigation in the app won't make the process stop as well I didn't test all the cases so I have no clue if there are other risky cases..

Please keep up the good work! ❤️

Hi I am new to this community. I tried qwen3-coder-30b-a3b but it was overthinking alot and not doing much of the work at all. I also tried Devstral-24b but I find it's quite a bit slow.

I'm running my model on a intel B70 arc pro on llama with sycl backend.

I'm testing my stack on a legacy c++ game including large librairie. My context window is quite large (~121k tokens) and never get filled fully anyway.

If you have some model proposition I'm quite open.

So what is the best agentic coding model for openclaude on for my set up?

hello first i want to say respect to unsloth devs they are doing a good job and i was impressed to see that unsloth studio has expanded to be more than fine tuning software,

i have an issue with the model loader it keep ignoring the context that i set and always load the default and when trying overriding from arguments sometimes it crash because i think it drops all the ui setting if you add only one argument, and i did make it work from argument but token generation dropped from 60 to 22 am i missing something or this is a bug

OS: kubuntu 24

Unsloth Version

v0.1.800-beta

Package Version

2026.8.18

Desktop App Version

0.1.800-beta

llama.cpp Version

b10360-mix-87da1a2

Hardware

GPU 0

NVIDIA GeForce RTX 5060 Ti · 16 GiB

GPU 1

NVIDIA GeForce RTX 5060 Ti · 16 GiB

CUDA

13.0

I do not understand how to get the model to limit reasoning. I am using unsloths q8 and q8_xl version in unsloth desktop. Combinations i have tried:
Low effort with a detailed prompt = 10+ minutes of thinking.
Low effort with a simple prompt = 10+ minutes of thinking.
Low effort with a simple prompt that requests a rapid prototype = 10 + minutes of thinking.
medium effort with simple prompt = 10+ minutes of thinking.

It seems like it always uses xhigh no matter what i do. However on random occasions it has thought for around 1 minute but i can't reproduce it.

Same issue when i connected it to hermes agent, and also happens on lmstudio but i dont even have effort level options in lmstudio so that is expected that it would default to xhigh.

system: windows 11, 1x tesla v100 16gb, 1x tesla v100 32gb, 32gb of ram, ryzen 3800x.
using mtp and originally used tensor parallelism but it would randomly give me issues and the api wouldnt respond so i disabled it. I get 56 tps with it off so you can judge "10 + minutes of thinking" appropriately.

I​'​'d like to stop using cloud models and do all of my vibe coding using local models instead. I currently use a EVGA GeForce RTX 2060 12GB​ XC Gaming GPU and have 32GB RAM​.

Which model(s) would be the best for me to work with for local code generation & edits using my current hardware setup? I am new to using Unsloth, so any configuration recommendations for me are appreciated also!​​

Project types include: Docker Compose, HTML, Java, CSS, Python, etc. I'd also like to use local models for general Linux support (Ubuntu, Fedora, server maintenance).

Thanks everyone!

OS Kernel Panic on M1 Max (32GB) when adjusting context size

**Environment:**

- Hardware: Apple M1 Max (32GB Unified Memory)

- Software: Unsloth Desktop

- Models tested: Qwen3.8-27B-UD-Q4_K_XL.gguf / Q3_K_XL.gguf

**Description:**

I would like to report a critical stability issue regarding context size adjustment in Unsloth Desktop.

When using `Qwen3.8-27B-UD-Q4_K_XL.gguf`, manually adjusting the context size even slightly causes a complete OS-level kernel panic (system crash). This has occurred twice.

- This issue does **not** happen when using `Qwen3.8-27B-UD-Q3_K_XL.gguf`.

- This issue does **not** happen if the context size is left to the default automatic configuration.

It would be highly appreciated if a safeguard could be implemented to prevent memory over-allocation that leads to system crashes.

**Additional Context:**

For comparison, this kernel panic never happens when using the `llama.cpp` CLI. The CLI safely rejects execution or fails to launch if the requirements exceed available memory. In fact, using `llama.cpp` CLI, the model runs successfully even with `CTX_SIZE="24576"`.