2x RTX PRO 6000 vs. 8x DGX Spark
Hi! The current price of the RTX PRO 6000 is almost double what it was a year ago, and it’s now about 3× the price of a single DGX Spark.
I currently have one RTX PRO 6000 and was considering buying another one. But at the current price, I’m wondering whether I should sell my existing RTX PRO 6000, sell my PC as well, and put some extra money toward a setup with 8× DGX Spark plus a good switch.
Has anyone actually tried an 8× DGX Spark setup? I’d really appreciate some advice, especially regarding the largest models you can run at a usable speed when working with a codebase.
Someone with a Mac Studio with 500 GB of unified memory told me that they can only use models around 200 GB in size. Anything larger is basically unusable—you can chat with it, but waiting for it to scan and reason over a codebase takes forever. They said the main bottleneck isn’t token generation speed, but prompt processing speed (prefill).
For those who have experience with large unified-memory setups or multiple DGX Sparks, what has your experience been like? What’s the largest model you’ve found usable for coding?