Login

r/threadripper

▲ 2 r/threadripper+1 crossposts

Rate my build Threadripper PRO 3975WX + Supermicro M12SWA-TF + 128GB DDR4-3200 + 2x Intel Arc Pro B70 32GB

  • CPU: 3975WX 32-core
  • Motherboard: Supermicro M12SWA-TF
  • 128GB DDR4-3200 (hoping to get 256gb ram by switching to 32gb sticks)
  • GPUs: 2x Intel Arc Pro B70

Got the CPU and mobo for $900.

I had most of the RAM sticks laying around from my past builds.

The B70s are still in the box; I paid $1k for each, but I'm thinking of just selling them and getting modified Nvidia cards like the 4080/4090 for CUDA.

Total is close to $3.5k.

Is this a good setup to run models and agents and learn how to manage a home AI setup?

I work in the software field as a forward deployed engineer, but I want to move into the AI field and become an AI forward deployed engineer, so I thought it would be beneficial for me to have the hardware at home.

I'm still waiting on the mobo and CPU to arrive. What do you think? It's not too late to return or sell some of this hardware and get something else.

reddit.com
u/Rokett — 6 days ago
▲ 1 r/threadripper

anybody run 96GB sticks on Asus WRX90?

Looking at this part number for a JEDEC kit: MTC40F204WS1RC64BC1

Not in the QVL, but JEDEC should be safe? Has anybody run this in 8x96 config?

reddit.com
u/sob727 — 6 days ago
▲ 6 r/threadripper

At my wits end with Threadripper WRX80

So I have been beating my head against the threadripper brick wall for several months now and don't know where to go from here. It all started with my Asus Pro WS WRX80E-Sage SE Wifi that had a 3955WX CPU which initially worked fine but then stopped posting after I plugged in three GPUs and never got beyond code 15 which is the start of DRAM testing. For that motherboard I replaced the PSU, RAM, And CPU and nothing changed so I concluded it must be the motherboard. I bought a brand new Gigabyte MC62-G40 and used one of the two 3955WX CPUs I have and RAM and PSU and now it refuses to POST. Once again I swapped out the CPU, RAM, and PSU and still no post. Unfortunately the Gigabyte MB provides no POST feedback but the board shuts down immediately. I have no idea where to go from here. Any suggestions would be very appreciated.

*** Update: I decided to return the board as defective to the vendor on eBay. I could flog this dead horse for a long time so I decided to end the madness. I do have these two 3955WXs and if I could find a MB that isn't such a death march to get working I would take another shot at it. Any suggestions on a decent WRX80 MB would be appreciated. Thanks all for the help. ***

reddit.com
u/Plugh_Xyzzy — 13 days ago
▲ 31 r/threadripper+1 crossposts

DDR4/PCIe4 vs DDR5/PCIe5 for LLMs- I benchmarked them for pre-training. What are your thoughts?

Hello everyone.

I've recently ran some experiments comparing DDR4/PCIe4 and DDR5/PCIe5 for AI workstations on a pre-training run, and would like to hear yours thoughts.

First of all, and as a summary of my results ( code here: https://github.com/Any-Winter-4079/DDR4-PCIe-4-vs-DDR5-PCIe-5-for-CUDA-training ), I rented two machines on Vast.ai, one with an H12SSL-i motherboard, an EPYC 7352, 192 GB of RAM and of course using PCIe4 (26.3 GB/s) and another with a WRX90E-SAGE SE motherboard, a 9975WX CPU, 256 GB of DDR5 RAM and PCIe5 (54.3 GB/s), and DDR5/PCIe5 is about 15-20% faster on pre-training (depending on whether you include or exclude validation and other costs) under the same number of GPUs.

With the current RAM prices, however, for the cost of 256 GB DDR5 RAM at 6400 MT/s you can get a full (extra) RTX PRO 6000 WS/Max-Q, at which point the comparison clearly favors DDR4/PCIe4 (with 2 GPUs), with about 50% extra throughput vs a single GPU at equal(ish) cost.

Now, there aren't a lot of downsides in my mind to choosing DDR4/PCIe4, but there can be a few:

  1. at least some of these DDR4/PCIe4 motherboards are on the older side, and were one to need replacement, they are not so easy to get (for example, the H12SSL-i used, I can only find it for sale as refurbished now, so who knows in a few years if it will even be available for retail).
  2. newer GPUs (as in, new NVIDIA generations) may stop working with older motherboards (meaning yes, PCIe is backwards compatible but the motherboard's BIOS/UEFI sometimes has issues during POST with newer GPUs (e.g., some older PCIe3 motherboards already have trouble recognizing Blackwell cards, and this may be the case for PCIe4 and newer cards in the future). Meaning if one were to buy a newer GPU in the future, the whole workstation may not be suitable.
  3. If one has to go ahead and bite the bullet on RAM prices, the million question is when? To me prices now are **awful** but so were RTX PRO 6000 prices and here we are (i.e., even higher).
  4. For pre-training, I would still choose DDR4/PCIe4, but I suspect for inference DDR5/PCIe5 might be a fair bit better than the 15-20% that it gives you on pre-training, plus we might be moving into some techniques soon such as dynamic expert/data loading into the model at runtime, which again would favor better DDR/PCIe speeds.

With all of this, I am curious if anyone has benchmarked this, and what are your thoughts on it. Would you hold out on DDR5 at the moment, and therefore go for PCIe4, or would you bite the DDR5 bullet early? Another issue with RAM is channels and DIMM count/channel, because if you want to go 'cheap' like let's only get 192 or 256 GB of DDR5 on 8 channels at 1 DIMM/channel (e.g., 8x32 to get 256), then upgrade to 512 later (when budget allows), that means you have to replace your full RAM (because all the slots are occupied, requiring new 8x64 to get 512 for instance)... And if you get fewer DIMMs like 4x64 to get 256 GB (leaving 4 DIMM slots unoccupied) then you get half the bandwidth because only 4 channels are populated. So maybe a machine that has dual DIMM support per channel is the answer to this (fully populating 8x32 to get the full bandwidth, and still allowing you to expand to another 8x32 to get 512), but in general it's a tricky point too.

So, what do you do/are you guys doing? Have you recently bought a workstation or upgraded to one, for pre-training, fine-tuning, RL, inference, whatever your use case may be, and come up with this dilemma? Are you choosing DDR4/PCIe4 as it would seem reasonable or are you going for DDR5/PCIe5 and if so, why? I am interested in all use cases and opinions!

u/Any-Winter-4079 — 14 days ago
▲ 6 r/threadripper

where do you guys get your RAM?

I finally decided to bite the bullet on 8x32 or 8x64 for a TR Pro build

But you cant really get that at the local Microcenter, and eBay server pulls are scary at that price (for transaction itself, warranty, etc). Vcolor is pricey.

How do you guys do it? Where do you get your DDR5 ECC?

reddit.com
u/sob727 — 14 days ago
▲ 3 r/threadripper

X399 with Threadripper 1920x not booting after force shutdown

Hello I recently build a pc for running local ai models with 4 mi 25s and I used a single fan header for cooling 3 of the cards with 40mm turbine fans but the software was monitoring only of the 3 cards and 2 of them were running a 32b model while the other 2 were idle and so they overheated and started beeping with their build in buzzer I panicked and shutted down the system. After that the system did not boot and stuck in some boot loop with the cpu light on i trough the bios is corrupt and i tried to reprogram it with an esp 32 but even though it recognized the chip it couldn't read or write on the chip. I also tried activating the dual bios but that did not work and I then shorted pins cs and gnd to get out of the loop and it actually got past that and got to code AC and A5 i think but it still did not boot and i think the cpu light was still on. Also i tried moving the single memory module and swapping it with another. I also used a gtx 1060 for video output since I did not want to deal with the flashed mi 25s. After the AC code it just was cycling between 1f and some other code for 10 min so i shut i down after that i stated it again but for some reason the aio pump was not working and the threadripper overheated and then pc shutted down. so now im stuck again with the initial c2 loop in the video. I also checked the visible choke for their voltages and they were .967v .947v 2.5v and 1.1v. Also on of the gpus uses its internal fan header to control its speed.

I'm coming to the the conclusion that the cpu is fried somehow but could it be something else what can I check?

I attached the video demo showing the issue. : https://drive.google.com/file/d/1Odtylz68Z9EYgBijZsZ-jfTZ8Xt1yHx9/view?usp=sharing

The exact specs are Threadripper 1920x, dual hyperx 32gb 4, mi25s flashed with wx9100 bios, Gigabyte auros x399 pro, corsair rm850, cooler master 750(powers 2 of the gpus and the fans, which are 130w total ).

reddit.com
u/tashanus — 13 days ago