u/kumits-u

Hi Guys,

I need a bit of advice, we're planning on procuring a server with 10x RTX 6000 PRO for local inference tasks.

I've configured a machine with config here

https://gpumachines.com/shared/asrock-20rack-204u10g-gnr2-2frf-2b-10x-6000-gpu-server-618320

Essentially it's 10x RTX 6000 Pro, but also with 2TB of RAM. I heard a rule of thumb of at least 2GB of RAM per 1GB of GPU VRAM. Now the question is - do I need that much RAM ? Cause we all know this eats up budget by a lot and I'd love to optimise the cost.

What do you think guys ? What's your experience ? Am I right saying that this rule of thumb is not entirely valid as it all depends on workload ?