Beginner looking for help building my first local AI homelab/server
Hi everyone,
I’m a complete beginner with AI homelab hardware and local LLM setups, and I’m trying to learn before buying anything.
My main goal is to build a home AI server for:
● Programming/coding
● Reasoning
● General-purpose AI/chat
● Learning AI/ML
● Local coding agents
I would like to experiment with local/open models from families such as DeepSeek, Qwen, Kimi, GLM, and other good coding/reasoning models.
I don’t need to run the biggest models available. I mainly want decent models that are actually useful for programming and reasoning.
Ideally, I would like to run one good model at a time, or possibly two models simultaneously if the hardware allows it.
I’m still learning what things like VRAM, quantization, multi-GPU inference, Ollama, llama.cpp and vLLM mean, so beginner-friendly recommendations would be really helpful.
What should I buy?
I’m open to both new and used hardware and would like the best performance for the money.
I’ve seen recommendations for things like:
● RTX 3090 24GB
● RTX 4090 24GB
● RTX 5090 32GB
● Dual RTX 3090
● Used Dell Precision / HP Z / Lenovo ThinkStation workstations
● Used enterprise servers
But I don’t know which direction makes the most sense for a beginner.
Could you please recommend a complete hardware setup, including:
● GPU
● CPU
● Motherboard
● RAM
● Storage
● PSU
● Cooling
● Case/rack
● Networking
● Any other hardware I would need
I’d also really appreciate recommendations for YouTube videos or beginner guides that show the complete process, starting from building/buying the hardware through installing Linux, NVIDIA drivers, Ollama/vLLM, and finally running local models.
If you were starting from zero today and wanted a local AI server mainly for coding + reasoning, what would you build?
I’m trying to learn this properly, so beginner-friendly explanations are very welcome.
Thank you!