Is this Threadripper Pro workstation worth it for local LLM inference and fine-tuning?
I am planning a new workstation for local AI work and general development. Main use case is local LLM inference, some LoRA / QLoRA fine-tuning, coding, containers, and some gaming on the side. This would not be for serving a public API, mostly just single-user local use.
My current system is a Ryzen 9 9950X3D build with 96GB DDR5, an RTX 5090 32GB, and fast NVMe storage. It is still a really good desktop, but I am starting to feel limited by system RAM, PCIe expansion, and bigger local model experiments.
The build I am considering:
CPU: AMD Ryzen Threadripper PRO 9985WX, 64 cores / 128 threads
Price: about $8,682
Motherboard: ASUS Pro WS WRX90E-SAGE SE
Price: about $1,447
RAM: 512GB total, 8 x 64GB Kingston KSM56R46BD4PMI-64HAI, DDR5-5600 ECC Registered RDIMM
Price: about $19,332
GPU: RTX PRO 6000 Blackwell 96GB
Already owned, not included in the quote
SSD: 2 x Samsung 9100 PRO 4TB PCIe 5.0 NVMe
Price: about $1,898
CPU cooler: Arctic Liquid Freezer WS360-SP6
Price: about $400
Case: Thermaltake AX700 full tower
Price: about $411
PSU: ASUS Pro WS 3000P, 3000W Platinum, ATX 3.1
Price: about $748
Case fans: 4 x Noctua NF-A14x25 G2 PWM chromax.black
Price: about $216
Total quoted price, excluding the RTX PRO 6000:
about $38,369
The main reason I am looking at WRX90 / Threadripper Pro is not just CPU speed. I want 8-channel ECC RDIMM, lots of PCIe lanes, a platform that can take another GPU later, and something that is stable for long-running workloads. For inference, I am especially interested in running models that fit fully in the 96GB GPU, but also larger GGUF / offload setups where the 512GB system RAM and memory bandwidth might matter.
What I am trying to sanity check:
- Does this build make sense for single-user local LLM inference with an RTX PRO 6000 96GB?
- Is the 9985WX actually useful here, or am I mostly paying for CPU cores I will not use?
- Any obvious compatibility issues with the WRX90E-SAGE SE, 8 x Kingston RDIMM, RTX PRO 6000, AX700 case, and 3000W PSU?
- For fine-tuning, would this platform be meaningfully better than a high-end consumer desktop, or is the GPU doing almost all the work anyway?
- Would you change anything before buying?
I know this is expensive and probably not the best pure price/performance setup. I am mostly trying to avoid spending this much and then finding out that I picked the wrong platform, wrong RAM configuration, or some annoying compatibility trap.