u/tricck3zz

I've been messing around with Qwen3.8 27B locally and I'm wondering if I'm getting the performance I should be getting or if my setup/config could be improved.

PC:

Ryzen 7 7800X3D

RTX 5070 Ti 16GB

RTX 3060 12GB

32GB DDR5-6000 CL30

Windows 11

llama.cpp / llama-server latest build

I'm currently running the Qwen3.8-27B UD Q4_K_XL GGUF with both GPUs using tensor split.

My current config:

llama-server.exe ^

-m "Qwen3.8-27B-UD-Q4_K_XL.gguf" ^

--alias "Qwen3.8-27B-UD-Q4" ^

--host 0.0.0.0 ^

--port 8035 ^

--n-gpu-layers 99 ^

--split-mode tensor ^

--tensor-split 60,40 ^

--main-gpu 0 ^

--parallel 1 ^

--flash-attn on ^

--cache-type-k q8_0 ^

--cache-type-v q8_0 ^

--ctx-size 131072 ^

--batch-size 2048 ^

--ubatch-size 512 ^

--threads 8 ^

--threads-batch 8 ^

--presence-penalty 0.0 ^

--repeat-penalty 1.0 ^

--temp 1.0 ^

--top-p 0.95 ^

--top-k 20 ^

--min-p 0.0 ^

--jinja ^

--reasoning-format auto ^

--no-mmproj-offload ^

--spec-type draft-mtp ^

--spec-draft-n-max 3 ^

--mmproj "mmproj-BF16.gguf" ^

--metrics

With MTP I'm getting around 40–46 tok/s depending on the run. I've seen around 41 tok/s pretty consistently, with n-max 3 seeming to be a little better than 2 for me.

Both GPUs are basically maxed during generation.

I'm mainly wondering:

Is ~40–46 tok/s reasonable for this hardware/config?

Is there anything obviously wrong or inefficient in my setup?

Would a different quant be a better choice for these GPUs? I've been looking at Ridge 3.7bpw, Q4/Q5 UD quants, etc.

Would it make more sense to use a smaller quant that could fit mostly/all on the 5070 Ti instead of tensor-splitting across both GPUs?

Is there anything I should change with the KV cache, batch/ubatch, tensor split, MTP settings, etc. to get better generation speed?

I'm mostly interested in coding/agent use through OpenCode, so I'd rather have a good balance of quality and speed than just chase the highest possible tok/s.

If anyone is running Qwen3.8 27B on a similar setup, I'd be interested to know what quant/config you're using and what kind of speeds you're getting.