I'm working on setting up Qwen 3.8 27B on my 4x3090 rig. I mostly used the config from Club3090 as a guideline.

One result I didn't expect was MTP significantly hurting performance at long context. My main usages is Hermes agent and if this is correct it suggests I should disable MTP all together or perhaps my settings are not optimal.

Benchmarks below - any thoughts?

Qwen3.8-27B-FP8 Comparative Benchmark Matrix

Speculative Setting |Short Prompt Decode (tok/s) |Short TTFT (s) |72k Long Prompt Decode (tok/s) |72k Long Warm TTFT (s) |72k Concurrency-2 Wall Time (s) |Steady VRAM / GPU
No MTP (Disabled) |67.5 |0.121 |57.3 |0.532 |15.25 |~20.5 GiB
MTP = 1 |80.5 |0.121 |15.6 |1.119 |48.05 |~20.7 GiB
MTP = 3 |106.3 |0.135 |22.6 |1.142 |32.08 |~21.2 GiB

I’m a longtime and very heavy Enterprise Cursor user.

I’m happy with the product but the unreasonableness of surcharge is blatant with new Luna pricing.

They charge a flat $0.25 surcharge on Tokens - it doesn’t sound like much but it includes cache hits.

For coding cache is 90%+ of token cost and cache costs should be 1/10th the surcharge.

So let’s look at Luna.

OAI costs 0.2 Inputs/ 1.2 Output / 0.02 Cache Reads.

Cursor surcharges charges over 100% for Input Tokens and 12.5x for Cache Reads!

It’s not specific to Luna. GPT Sol only charges 0.50 for Cache reads somehow - so Cursor has a 50% mark-up here too.

Why are enterprise clients accepting this? I’m happy for Cursor to make a profit but this is too much.

I managed to setup a 4 x 3090 server on a WRX80 Threadripper server board. It’s running Ubuntu Sever and vLLM for model hosting.

My primary use case is a Hermes AI assistant.

I currently run Qwen 3.6 27B with q6 weights and 250k context full precision TP4 and feel it isn’t the optimal setup. I’m not maximizing VRAM or model accuracy. Avoiding FP8 quants because of Ampere is also a headache.

What are others running?