u/Weeblewobbly

Just finished setting up my macpro 2019 as an LLM server. This a 12 cores, 96Gb ddr4, 2Tb, and more importantly Radeon Vega II 32gb vram.

On the software side, I'm running a headless Nixos server with llama.cpp.

So far, I'm impressed: qwen3.6-27B-UD-Q5_K_S runs at a rock steady 26-27 tk/s which I consider very usable after sluming below 10tk/s for the same dense model at Q3 on MacBook M4 32Gb ram. That's the only model I've tested so far. At 19Gb, it leaves plenty of room for kv cache.

I expect the area of tinkering will be to find the best combination of dense model size Vs kvcache room, and then of course testing some MoEs.

My plan is to run it as a backend for pi.dev, and to serve the rest of the household with a chat interface running on my proxmox server.

If you are lucky enough to have one of these beats lying around, you could do worse than turning them into an LLM server.

This is probably quite niche, but someone cleverer than me might have cracked it already.

I spent a couple of hours diving in the SDK to see if there's anyway to view, upload and download files on, to and from a SharePoint document library that's been setup in the integration as a file integration to a dataverse table.

I went as far as adding the virtual table that joins the records with the SP folders, but could not construct a odata request that would return anything meaningful other than the SharePoint location.

It looks to me that the only way forward is using flows, but curious to hear if anyone has looked into it.