▲ 15 r/AIProgrammingHardware+1 crossposts

Best local LLM for OpenCode at 40+ tok/s with 2× RTX PRO 6000 Blackwell?

I'm looking for the best open-weight LLM I can realistically run locally for OpenCode, with a target of at least 40+ output tokens/sec while using the model's full context window.

My workstation:

  • AMD Threadripper PRO 9985WX, 64C/128T
  • 512 GB DDR5-5600 ECC RDIMM, 8-channel
  • 2× NVIDIA RTX PRO 6000 Blackwell 96 GB (192 GB total VRAM)
    • 1× Workstation Edition
    • 1× Workstation Max-Q
  • ASUS Pro WS WRX90E-SAGE SE
  • Linux
  • Mainly using llama.cpp / LM Studio, but I'm also open to vLLM or SGLang if they make more sense

The main use case is agentic coding through OpenCode, so I'm prioritizing coding ability, tool use, long-context reliability, instruction following, and avoiding repetition/loops.

I'm fine with FP8, NVFP4/MXFP4, GGUF, etc. if the quality trade-off is reasonable. The model does not necessarily have to fit entirely in VRAM; CPU/RAM offloading is also an option, but I still want 40+ tok/s generation speed at full context.

I'm basically looking for the smartest model this machine can run at that speed, rather than the fastest small model.

What would you pick today?

I'd especially appreciate actual benchmarks from similar dual-96GB Blackwell setups rather than theoretical estimates.

reddit.com
u/mrblithe — 2 days ago

Made a free/open-source online tool for tracking your own CPAP data, looking for feedback

I was diagnosed with sleep apnea not long ago. After that, I got a CPAP machine and started treatment. When I wanted to look at the data from my CPAP, I found OSCAR, an open-source tool that does a genuinely great job and that I still highly recommend.

The problem was that checking my data always meant opening my computer, and sharing it with my doctor meant either carrying the SD card around or sending screenshots. I also couldn't conveniently look at my nights from my phone, which kept bothering me.

Later, I started looking into alternatives and found a few cloud-based apps that work similarly to OSCAR. You upload the data from your SD card and they process it and present the results online.

The problem was that most of their free tiers were extremely limited, while the versions that were actually useful started at around $20 per month. So I decided to build this tool instead, both for myself and to make it available to all of you for free.

I don’t want this to come across as advertising, especially since I’m going to release it completely free and open source. My only goal is to help people dealing with similar issues, test it with more devices, and gather feedback so I can keep improving it.

---

So; introducing PapSee:

You upload what your machine wrote to the card, it parses it and gives the night back: flow waveform, every scored event, AHI, leak, pressure, usage, plus history and trends across nights, on whatever device you happen to be holding.

Also you will ability to self host on your own if you don't want to share your health data. But if you want to try now I'd really like is feedback.

Try it with your own card and tell me where it's wrong, confusing, or missing something you'd expect from OSCAR. Non-AirSense-11 cards especially, because that's the one thing I can't test myself. And if your answer is "I would never upload this data to a website", that's a completely fair position and I'd genuinely like to know how common it is.

There's an example patient you can click through if you just want to see the screens without signing up or uploading anything of your own.

Roadmap I'm thinking about: more device brands (Philips is the obvious next one, though I need real sample cards for that), a mobile app, and importing sleep data from Apple Health.

WebSite: https://www.papsee.com

GitHub: https://github.com/mr-blithe/papsee

>For those without a technical background, open source means that the source code is publicly available. In other words, you do not have to use www.papsee.com to use this program. You can host it yourself on your own computer or on platforms such as Vercel. You can also review all of the source code and modify it however you like.

reddit.com
u/mrblithe — 10 days ago

Do I likely have sleep apnea based on my Apple Watch data?

Hi everyone,

My Apple Watch recently gave me a Sleep Apnea Notification based on my breathing disturbances data collected between June 8 and July 8.

Here are my breathing disturbance graphs (1 month and 6 months).

A few details about me:

  • Male, 34 years old
  • BMI around 33

I know the Apple Watch can't diagnose sleep apnea, but I'm wondering how concerning these graphs look to people who have been diagnosed.

Do these results look like they could indicate obstructive sleep apnea, or have any of you received similar Apple Watch notifications that turned out to be false alarms?

I'm planning to get a sleep study, but I'd appreciate hearing about your experiences while I wait.

Thanks!

https://preview.redd.it/t9ljhje4kuch1.png?width=707&format=png&auto=webp&s=5d28cea1514f85ecc8e494dce22c75fc069101bf

https://preview.redd.it/ocaiaql5kuch1.png?width=707&format=png&auto=webp&s=916634cef799b0278c23c501c1223de301bb9af8

reddit.com
u/mrblithe — 1 month ago

Is this Threadripper Pro workstation worth it for local LLM inference and fine-tuning?

I am planning a new workstation for local AI work and general development. Main use case is local LLM inference, some LoRA / QLoRA fine-tuning, coding, containers, and some gaming on the side. This would not be for serving a public API, mostly just single-user local use.

My current system is a Ryzen 9 9950X3D build with 96GB DDR5, an RTX 5090 32GB, and fast NVMe storage. It is still a really good desktop, but I am starting to feel limited by system RAM, PCIe expansion, and bigger local model experiments.

The build I am considering:

CPU: AMD Ryzen Threadripper PRO 9985WX, 64 cores / 128 threads
Price: about $8,682

Motherboard: ASUS Pro WS WRX90E-SAGE SE
Price: about $1,447

RAM: 512GB total, 8 x 64GB Kingston KSM56R46BD4PMI-64HAI, DDR5-5600 ECC Registered RDIMM
Price: about $19,332

GPU: RTX PRO 6000 Blackwell 96GB
Already owned, not included in the quote

SSD: 2 x Samsung 9100 PRO 4TB PCIe 5.0 NVMe
Price: about $1,898

CPU cooler: Arctic Liquid Freezer WS360-SP6
Price: about $400

Case: Thermaltake AX700 full tower
Price: about $411

PSU: ASUS Pro WS 3000P, 3000W Platinum, ATX 3.1
Price: about $748

Case fans: 4 x Noctua NF-A14x25 G2 PWM chromax.black
Price: about $216

Total quoted price, excluding the RTX PRO 6000:
about $38,369

The main reason I am looking at WRX90 / Threadripper Pro is not just CPU speed. I want 8-channel ECC RDIMM, lots of PCIe lanes, a platform that can take another GPU later, and something that is stable for long-running workloads. For inference, I am especially interested in running models that fit fully in the 96GB GPU, but also larger GGUF / offload setups where the 512GB system RAM and memory bandwidth might matter.

What I am trying to sanity check:

  1. Does this build make sense for single-user local LLM inference with an RTX PRO 6000 96GB?
  2. Is the 9985WX actually useful here, or am I mostly paying for CPU cores I will not use?
  3. Any obvious compatibility issues with the WRX90E-SAGE SE, 8 x Kingston RDIMM, RTX PRO 6000, AX700 case, and 3000W PSU?
  4. For fine-tuning, would this platform be meaningfully better than a high-end consumer desktop, or is the GPU doing almost all the work anyway?
  5. Would you change anything before buying?

I know this is expensive and probably not the best pure price/performance setup. I am mostly trying to avoid spending this much and then finding out that I picked the wrong platform, wrong RAM configuration, or some annoying compatibility trap.

reddit.com
u/mrblithe — 3 months ago

Is this Threadripper Pro workstation worth it for local LLM inference and fine-tuning?

I am planning a new workstation for local AI work and general development. Main use case is local LLM inference, some LoRA / QLoRA fine-tuning, coding, containers, and some gaming on the side. This would not be for serving a public API, mostly just single-user local use.

My current system is a Ryzen 9 9950X3D build with 96GB DDR5, an RTX 5090 32GB, and fast NVMe storage. It is still a really good desktop, but I am starting to feel limited by system RAM, PCIe expansion, and bigger local model experiments.

The build I am considering:

CPU: AMD Ryzen Threadripper PRO 9985WX, 64 cores / 128 threads
Price: about $8,682

Motherboard: ASUS Pro WS WRX90E-SAGE SE
Price: about $1,447

RAM: 512GB total, 8 x 64GB Kingston KSM56R46BD4PMI-64HAI, DDR5-5600 ECC Registered RDIMM
Price: about $19,332

GPU: RTX PRO 6000 Blackwell 96GB
Already owned one, not included in the quote

SSD: 2 x Samsung 9100 PRO 4TB PCIe 5.0 NVMe
Price: about $1,898

CPU cooler: Arctic Liquid Freezer WS360-SP6
Price: about $400

Case: Thermaltake AX700 full tower
Price: about $411

PSU: ASUS Pro WS 3000P, 3000W Platinum, ATX 3.1
Price: about $748

Case fans: 4 x Noctua NF-A14x25 G2 PWM chromax.black
Price: about $216

Total quoted price, excluding the RTX PRO 6000:
about $38,369

The main reason I am looking at WRX90 / Threadripper Pro is not just CPU speed. I want 8-channel ECC RDIMM, lots of PCIe lanes, a platform that can take another GPU later, and something that is stable for long-running workloads. For inference, I am especially interested in running models that fit fully in the 96GB GPU, but also larger GGUF / offload setups where the 512GB system RAM and memory bandwidth might matter.

What I am trying to sanity check:

  1. Does this build make sense for single-user local LLM inference with an RTX PRO 6000 96GB?
  2. Is the 9985WX actually useful here, or am I mostly paying for CPU cores I will not use?
  3. Any obvious compatibility issues with the WRX90E-SAGE SE, 8 x Kingston RDIMM, RTX PRO 6000, AX700 case, and 3000W PSU?
  4. For fine-tuning, would this platform be meaningfully better than a high-end consumer desktop, or is the GPU doing almost all the work anyway?
  5. Would you change anything before buying?

I know this is expensive and probably not the best pure price/performance setup. I am mostly trying to avoid spending this much and then finding out that I picked the wrong platform, wrong RAM configuration, or some annoying compatibility trap.

reddit.com
u/mrblithe — 3 months ago