The cyber shelf - 4x 16gb home lab
Been a long time lurker of this subreddit, learned a whole lot from here and Gemini. I've finally got my rig somewhere I feel I could share. Lot of people talk about racks for their home lab but all I managed was this kitchen rack. I was just dipping toes in the water with my first 16gb card and just ended up stacking them. This is my 4x 16gb card build (bifurcated main slot, riser cable on one pcie3 1x slot that runs two llama.cpp instances of qwen 3.6 spec decoding q4_0 with one context train of 150k each, 1000 tok/s prompt processing, 45-60tok/s generation. I5 processor with 32gb ddr4, but I'm all on vram. Used opencode to build up the backend that does the llamacpp management and token counting. If these calcs are right (haha no idea really) says here I've saved 60 bucks already! Everything is buggy as hell but that's a skill issue on my end. Was trying to build a router so I could run a parallel 2 on one set of cards and run a parallel 1 on the other set, then forward them to the right server and that's where I am now. AMA or leave a (mean) comment or suggestion!