Login

u/Possibility_22

▲ 6 r/LocalLLM

RTX Pro 4000 Blackwell - a good option for starting out with local LLMs?

I recently purchased an RTX Pro 4000 Blackwell SFF for a small form factor PC build I was putting together. At the time it was available at around the same price as a regular RTX Pro 4000 Blackwell, ~£1,700 (or so I thought).

As far as I know the SFF card is pretty much the same as the standard one except it has a lower TDP (70W - no external cable needed - vs 140W) and is a two slot, low profile card instead of a single slot regular height card.

I've now seen that Lenovo are selling the regular RTX Pro 4000 Blackwell for under £1,300: https://www.lenovo.com/gb/en/p/accessories-and-software/graphics-cards/graphics_cards/4x61t95636 (that includes a 2% discount you get by checking a box). Other stores still seem to be selling the same card for around £1,700 to £2,000.

Is this a good deal or is the card just over priced elsewhere? It makes the SFF card I purchased seem quite expensive. I'm a software developer and I was thinking about starting to get into local LLMs - are these cards a viable option?

I've seen that the Radeon AI PRO R9700 is also available for under £1,300 and has 32GB VRAM vs the 24GB the RTX 4000 has. It's a two slot card that uses a lot more power though. Would that (or something else entirely) be a better option?

reddit.com
u/Possibility_22 — 19 hours ago
▲ 14 r/sffpc

UPDATE: First SFF build - total disaster - seeking advice on what I can/should salvage

Original Post: https://www.reddit.com/r/sffpc/comments/1qetrsa/first_sff_build_total_disaster_seeking_advice_on/

TLDR: Changed a lot of components, much bigger case - have a (somewhat) working PC which is still technically SFF with a few quirks.

Thanks for all the advice in response to my original post. I took some time away from the build but did eventually come back to it and thought I should post an update. I changed out a lot of components so it might not even technically be considered the same build. The following components are what I originally intended to use:

Original components used:

  • Motherboard: Asus Pro H610T-CSM DDR5 (thin mini ITX)
  • CPU: Intel Core i5-14500T (35W)
  • RAM: Crucial 64 GB (2 x 32 GB) DDR5-5600 SODIMM CL46
  • Network: WiFi 7 BE200 M.2 Key E Network Card
  • Storage
    • Samsung 860 EVO mSATA SSD 250GB (used)
    • Team 2TB MS30 M.2 SATA

New components added:

I've also had to use the following extra components/adapters to get everything hooked up:

  • M.2 to PCIE x16 riser cable
  • PSU cables:
    • 2 pin to 4 pin ATX
    • 4 pin to 8 pin ATX
  • mSATA to SATA adapter
  • M.2 SATA to SATA adapter
  • Toggle switch with jumper cables
  • USB 3.1 Gen 1 header to dual USB 3.1 Gen 1 headers
  • USB 3.1 Gen 1 to USB 3.1 Gen 2 (USB E) header
  • Lots of zip ties, rubber fan fasteners

Build

I had decided to change the case. Initially I thought I'd go for a similar but larger, sub 4 litre case like the one I started with. I attempted to use a CiT S2i but that basically ended up with the same issues. In the end I changed to a much larger (but still SFF) case that I found on Amazon (around 8 litres I think). This allowed me to use a standard (for a SFF build) CPU cooler and generally gave mor space for the components.

I did discover a few issues with my use of of a thin mini ITX board without a PCIe slot. Where the PCIe slot would normally be there were electrical components (resisters and things maybe) which got in the way of the riser cable's slot. I therefor needed to use a low profile GPU and have it "floating" at the top of the full height slots. The GPU I was going to use was low profile but attaching it became an issue as there seems to be different alignment between the screw holes and GPU for low profile and full height cards. I had to move to a two slot card with a central attachment hole to get around this. Given that I was limited in how I could power the card (absolute max of 70W) as well as by the dimensions of the case, I ended up going with the Arc Pro B50.

Having all of the extra space in the new case made it seem silly to have a large external power supply as well. I decided to go with an internal solution. I chose an HDPLEX 250W GaN ATX PSU to save space in the case. However, the motherboard works with 19V input and ATX PSUs like the HDPLEX supply 12V. I eventually found a suitable buck-boost DC-DC converter to convert the output of HDPLEX to 19V. Side note - it was very difficult to find the cables to power the board from the internal two pin ATX connector. In the end I had to use a combination of cables I found on Amazon and MODDIY. It seems like the thin mini ITX format hasn't really taken off, for consumer products at least.

I then had to attach all the components. Since they storage was not standard (an mSATA SSD and an M.2 SATA SSD, each in a separate SATA adapter) I ended up attaching them to the sides of the interior of the case with zip ties. I did the same with the Mini-Box converter and managed to add in a case fan using some rubber attachment things. At this point I thought I was done, the machine turned on and seemed to work. However, there proved to be a significant issue.

The motherboard, CPU and storage was all powered directly from the 19V Mini-Box converter. Unfortunately, that has a maximum output of 160-180W. To try to avoid issues with this limitation, I had (and still have) the GPU powered directly from the HDPLEX PSU (via SATA power to the riser cable). To get the HDPLEX PSU to turn on without being connected directly to the motherboard I was using a 24 pin ATX adapter which effectively mimicked the PS-ON signal (grounded a pin?).

This was fine when the computer was running but when I shut it down I found that the GPU fans would spin up to maximum and just continue on. The only way to power down the GPU was to unplug the power cable - not ideal. I think this was because the HDPLEX was still providing power (the pin was still grounded) but there was no signal from the motherboard to sort of regulate anything. At that point I started a process where I bought lots of random cables and circuit boards and hooked them all up in various ways. None of it worked.

I'd almost given up when I realised that I could hook up the power switch header on the motherboard to the Mini-Box converter. This sort of helped things but it required that the HDPLEX was powering the converter for the switch to work in the first place. I managed to make it so that holding down the power switch would start up the system. I needed to keep it help down though which triggered a system shut down after a few seconds - the power switch is meant to be a momentary signal. After experimenting a bit more I found a workable(ish) solution.

It turns out the HDPLEX has a header so that it can be used in conjunction with other HDPlex units to provide more power. I managed to hook this header up to a toggle switch using some jumper wires. I then replaced the case's on/off switch with the toggle switch. Turning the system on works as it grounds the pin (or something like that) which starts up the HDPLEX which powers up the Mini-Box which sends the PS-ON signal to the motherboard.

Turning off is still a bit of an issue as flicking the toggle switch is effectively the same as pulling out the power cord - not something you want to do regularly. Turning the PC off via the OS leads to the same issue with the GPU fans spinning up. Essentially powering off the system is a two stage process, first via the OS and then flicking the toggle switch as the GPU fans start to get loud. It's more of a work around to the problem than an actual fix I guess. It is much easier to flick a switch that unplug and plug in the power cable all the time though. The PC going to sleep causes the same issue with the GPU so I've turned off the functionality for that to happen automatically after a period of time.

I also scratched the front of the case quite badly when securing the toggle switch - I can always put a sticker over that though.

Worth it?

At the time I bought it, I think I saved approximately £50 by getting SODIMM memory rather than regular DDR5 DIMMs (which would have likely had a better spec). However, I think the components and adapters I ended up needing to use due to that decision probably cost hundreds of pounds more than components I could have used if I hadn't gone with the SODIMM memory.

More standard components would also probably perform better than what I ended up using. Also, because I didn't know what I was doing (still don't) I ended up buying probably over a dozen cables and other components which I didn't need in order to try to solve the issues with the power supply and GPU.

The GPU also doesn't seem to perform brilliantly when you plug a cabler directly into it so I've been using the HDMI port on the motherboard which seems more stable. That may be due to to the PCIe 4.0 x 4 M.2 slot the GPU is attached to (via riser cable). It could also be because I was using a mini DisplayPort to DisplayPort adapter on top of a DisplayPort to HDMI adapter and HDMI cable (I was testing using a display without a DisplayPort input).

If you do want to buy SODIMM RAM and use a dedicated GPU I would suggest going with a motherboard which can use a standard 12V ATX power supply rather than a 19V thin mini ITX board. I've used the ASUS PRO Q870I-C-CSM Intel Q870 in another build and it was much easier, all things considered.

u/Possibility_22 — 11 days ago
▲ 6 r/IntelArc

Hi, I recently purchased a Sparkle Intel ARC Pro B70. I'm having trouble getting any signal out of it and I'm wondering if it is just a defective card.

I am using it with:

Motherboard: GIGABYTE Z890I AORUS ULTRA

CPU: Intel Core Ultra 5 250K Plus

PSU: Corsair SF Series SF850 Fully Modular 80 PLUS Platinum SFX Power Supply (UK)

RAM: Corsair Vengeance DDR5 96GB (2x48GB) 5600

Riser cable: PCIe 5

Operating system: Windows 11 Pro

Without a GPU installed I can access the Bios and have enabled XMP and resizable Bar. Windows boots fine and is usable. I have downloaded and tried to install the latest Arc Pro driver.

With the B70 installed, the fans spin up and I can get to the GIGABYTE splash screen and enter the BIOS. When I try to get to Windows the screen remains black. If I tap the keyboard it will sometimes flash. This happens when the HDMI cable is still plugged into the motherboard.

At first I thought the riser cable might be damaged as I noticed a small deformation in it. I've bought a new riser cable and exactly the same thing happens.

I've now tried swapping in a Nvidia RTX 5080 card which takes the same power connection (12vhpwr) and everything works as expected, no issues that I have noticed.

I'm thinking the card itself must be the problem but I was wondering if I could have missed anything in BIOS or something like that.

u/Possibility_22 — 19 days ago