Lost in a sea of suggestions!
New developer here - I’ve been looking to upgrade an old laptop (4GB VRAM, 16GB RAM, 500GB SSD) and will be moving to a desktop setup and remoting in with the old laptop using Sunlight/Moonlight. I’ll mainly be working with small product specific toy LLMs for engineering drafts and 3d modeling/rendering. There will also be some LLM training as I go for additional task specific models. I’ve been reading reviews and reddit posts regarding a minimum setup required for my work and I had settled on a 5070ti, 16GB VRAM, 32GB RAM (2 - 16 sticks) with a 2TB SSD to start with (can upgrade as I grow). However, I’m in a few groups where they’ve commented on other posts that a setup like this is no longer adequate for such work because of the possibility of needing to run a larger LLM for reference. If I’m sticking to LLM models less than 10M, shouldn’t this setup work fine? They all seem to think a $10k+ rig with a 5090, 32 VRAM and + 96 RAM is required which is absolutely not possible. Please tell me I’m not crazy…