Self-hosted Qwen3.8-27B on 2× RTX 4080 Super ( 2 x 32 GB VRAM) — 152 tok/s, ~1 EUR/hr
Just got Qwen3.8-27B (FP8) running on rented GPUs from Trooper AI. FP8 fits on 64 GB VRAM with decent results.
Stack:
- GPU: 2× RTX 4080 Super Pro (64 GB VRAM total)
- CPU: 12 P-cores, 76 GB RAM
- SSD: 900 GB NVMe
- Price: 1.06 EUR/h
Served it via vLLM → KServe → Envoy AI Gateway, TLS + token metering + rate limiting on top ( I already have a running Kubernetes cluster, I attached to trooper GPU node), or you can it serve it directly with compose if you are a single user.
Runned load tests: (10 concurrent requests, 32768 context)
- TTFT: ~0.9s
- Per-stream decode: ~28 tok/s
- Aggregate: 152 tok/s
Full deploy guide if you want to deploy it: https://github.com/redaER7/qwen3.8-27b-self-hosted
Now, looking to deploy the full model FP16 on RTX 6000 Pro
u/Former_Squirrel_2726 — 9 hours ago
