Qwen3.6-27B at 32 concurrent clients: 697 ms TTFT with Paddock vs 2.5 s (vLLM) / 6.9 s (llama.cpp) on one Blackwell card, and it already runs Qwen3.8
Disclosure first: I'm one of the developers of Paddock (Truespar). It's a new inference engine, free for individuals and companies with no usage limits, and these are our own published benchmarks, so be skeptical and reproduce them if you can.
Numbers from the public bench (RTX PRO 6000, byte-identical official FP8 checkpoint, spec decoding off everywhere, max batch 32, ctx 8192).
Throughput, tok/s at 1 / 8 / 32 concurrent clients:
- Paddock (FP8): 47.7 / 334.1 / 968.1
- vLLM (FP8): 45.0 / 321.1 / 968.1
- SGLang (FP8): 45.0 / 321.9 / 835.0
- llama.cpp (Q8_0): 33.7 / 64.9 / 239.1
Time to first token at 32 clients:
- Paddock: 697 ms
- SGLang: 1.9 s
- vLLM: 2.5 s
- llama.cpp: 6.9 s
Full methodology: https://truespar.com/paddock/benchmarks/qwen36-27b. Paddock exposes a plain OpenAI endpoint, so you can throw NVIDIA's aiperf at it on your own hardware.
Since the Qwen3.8-27B drop we've been optimizing kernels for the whole 3.x generation, and 3.8 runs today: if you have a supported Blackwell or Ampere card (list in the docs) I'd love to see your 3.8 numbers before we publish our own bench for it.
Two things beyond raw speed, since the built-in Studio is half the product: drop a document on it and it runs an integrated Docling-style pipeline (segmentation, tables, reading order) plus tag-based extraction with Granite 4.1 Vision, tables to JSON/CSV straight from page images, dozens of pages in a single inference with Unlimited-OCR. And images get EXIF/metadata enrichment before the model sees them, so a local Qwen knows when/where a photo was taken without any tool calls. You can also A/B a local model against OpenRouter models side by side.
One dependency-free download (Windows/Linux x64, NVIDIA driver 580+), no Python stack. Download + docs: https://truespar.com/blog/introducing-paddock
I'll be in the comments