GPU Recommendation for Homelab
Buying my first dedicated inference GPU for an existing Proxmox homelab. I've narrowed it to three cards and keep going in circles, so I'd rather hear from people actually running them. I am very new to local AI and am mostly interested in this purchase to learn. Budget is up to $1500 for the card.
The box
Ryzen 7 9800X3D / ASUS ROG STRIX X870E-E / 32GB DDR5-6000 (two slots free) / Corsair HX1000i 1000W / PCIe 5.0 x16 available / Proxmox VE 9.2, kernel 7.0.14-11-pve.
Two things that matter more than the specs:
- It's on 24/7 and already runs my whole homelab, so idle power and driver stability are important.
- Everything runs in unprivileged LXC containers (today), not VMs. So passthrough for me means driver on the Proxmox host (correct me if I'm wrong when it comes to having a dedicated GPU here). The iGPU currently handles Jellyfin transcoding this way.
What I want to run
Mainly local LLM inference - general chat, coding help, and a homelab agent that monitors/reads logs and configs and suggests fixes. Secondary: Immich CLIP search and face recognition and possibly implementing one of the Paperless AI solutions.
Speed is not super important to me, but I'd like enough responsiveness that I'm not losing patience for a simple inquiry. The vast majority of my use case is monitoring services across Proxmox, my UNRAID NAS, a backup NAS, and my Unifi-based network. I'm assuming this is a fairly light load that smaller models can handle, but also want an LLM to be able to help me diagnose and troubleshoot.
The options
| Used RTX 3090 | RX 7900 XTX | Radeon AI PRO R9700 |
|---|---|---|
| VRAM | 24GB | 24GB |
| Bandwidth | ~936 GB/s | ~960 GB/s |
| TDP | 350W | 355W |
| Price | ~$1,000–1,100 used | ~$700–900 |
| Stack | CUDA | ROCm |
My questions
1. Is 32GB at 640 GB/s actually better than 24GB at ~940 GB/s here? This is the whole decision. The R9700 fits a bigger model; the other two are faster on whatever fits. For a mixed agent + photo-ML workload, which way does that fall in practice?
2. How much CUDA-only pain am I avoiding by paying the NVIDIA tax?
3. Anyone running ROCm on a current Proxmox kernel? I'm on 7.0.14-11-pve.
4. Idle power, real numbers? With a model loaded and sitting there vs unloaded.
One AMD-specific worry: my iGPU is /dev/dri/renderD128 and that path is passed into my Jellyfin container. A second AMD card means two render nodes with no guaranteed stable numbering. A udev rule pinned to the PCI address might fix it, but is that a non-issue in practice, or a recurring annoyance?
Thank you in advance. I'm happy to answer questions about the setup.