Upgraded to 2x RTX Pro 6000
With the dense Qwen 3.5 / 3.6 models, I've been amazed it has become possible to run almost frontier models locally, on prosumer hardware. I ended up switching my 5090 for a pair of 6000 workstation so I could try larger models.
After some pain with vllm and the realization that I don't actually own real Blackwell cards (consumer GPUs don't use the same architecture as datacenter ones), I got it to run DeepSeek V4 Flash at 80-100 tok/s, with full context and some room for KV cache (+4M with L2 cache). It's definitely a step up from the dense Qwen models for the tasks I have tried so far, mostly research, coding. It feels good enough to replace about 80-90% of 5.5/Opus sessions.
Given how cheap inference on Deepseek models is, it is of course not "worth" it, but it still hits different hearing the coil wine of your build as it is spitting a self contained html for the hundredth time. (Or the smoke detector going off because you missed setting the power limit on the GPUs, and the sudden 1.2KW load triggered big voltage drops on the line)