u/HyperWinX

Hello.

So, first, ill probably explain the context, and why we (well, I kinda have a team, so ill say we) want to build something like that.
We live in Russia. There are many preconditions, that point to the fact that pretty soon, we'll get our internet access (global internet, apparently) completely cut off. And we are preparing to go fully offline for indefinite period. I know, this sounds weird. No one believes that. But I prefer to be ready for the worst case scenario. ...and we kinda want to have our own AI rig.

Regarding the rig. Estimated specs:
- Mobo: ZX-DU99D4 v1.31
- CPUs: 2x Intel Xeon 2683v4
- RAM: 8x 32GB DDR4 ECC 2133 (saturates four channels, 2x per CPU. estimated bandwidth - 63GB/s, according to benchmarks posted here a while ago).
- GPU: we are kinda hesitant about this one. But it's a fact that if we want to make it at least somewhat usable, we need one. I stopped at V100 16GB, as one of the cheapest options for 16GB (on our used market). I know that with CUDA is barely supported, and not worth the hassle, but Vulkan exists, right?

Converting to USD - ~$1500. Not bad. Strix Halo boxes cost much more, and offer much less RAM, though the bandwidth and overall efficiency is much better. But we'd like to start with cheapest option possible. We definitely will add GPUs to the rig later.

We are targeting models like Deepseek V4 Flash and MiMo V2.5. Estimated performance:
- DSv4 Flash: 284B A13B model, UD-IQ4_XS. ~7GB per token bandwidth, sooo best case performance is 9tps, though, adding the context, it'll be more like 6-7tps. Prompt processing somewhere at 50-60 without GPU. Hopefully, with GPU it will go at least to 200-300tps.
- MiMo V2.5: 310B A15B model, same UD-IQ4_XS. 1-2 tokens slower than Deepseek. not mentioning the prompt processing

I've searched through this sub on tips for improving CPU only throughput, and collected them all here:
- Use ik_llama.cpp instead of llama.cpp.
- Configure batch / ubatch.
- OMP_WAIT_POLICY=ACTIVE env variable.
- Use MTP model.

Experimented a bit on my main rig with a smaller model - Qwen3.5 9B at Q6_K.
The only change that brought meaningful change - ik_llama.cpp. It doubled prompt processing speed, and added 1tps to tg. (best speed I got - 53tps pp, 4.6 tps tg). That's pretty much all. Pretty sure MTP will improve the situation, but llama-bench doesn't test that.

What would you say about that? What else can we do about performance / hardware? I also worry about interprocessor communication being slow, and therefore causing much less real bandwidth.

Thank you for reading this.

UPD: while I was writing this post, I found out that each CPU has 4 memory channels. Need help finding a cheap mobo that can give 8 channels total...

Hello! So, basically title.

Hardware specs:

CPU: AMD Ryzen 5 5600G

GPU: AMD RX 580 8GB (not 2048SP - just a regular 580)

Mobo: Asrock B450M Steel Legend

Displays: two Xiaomi displays, one is 4K, the other one - FHD. I'd like to connect 4K over DP, and FHD over HDMI.

What happens: the chance of DP working is really low. Usually, it starts fine, OpenCore (with driver for pretty boot menu) detects the 4K display, scales fine, then macOS's boot logs display on it correctly. When it goes to the lock screen - DP is "active", but there is no display. The second one works. If I'll try to reconnect it multiple times it *might* work. But usually it doesn't.

All my kexts

config.plist

I searched on YT, read dortania's guide on connector patching (post install guide) but I didn't understand / find anything related to my hardware

No OC Simplify involved.

By the way, I'd really love to migrate to my iGPU. If anyone can help me with NootedRed setup - id be really grateful. I tried installing the kext and doing everything listed in prerequisites, but it didn't boot.

Hello everyone. Yesterday i installed Hackintosh on my main PC for the first time. Surprisingly, it wasnt difficult at all, and i spent like ~7 hours, including waiting for the first installation phase two times, and passing OOBE on HDD. Then, I made sound work with VoodooHDA. And finally i started doing research on virtualization software, and found out that my only options are VMware Fusion 10 or Virtualbox 6.1.2 (i genuinely hate it), and even then i wont get any GPU acceleration. Is all that info correct? As far as i know, i cant even run Docker, which sucks.

День добрый! Надеюсь у вас все хорошо, ибо даже у меня, в кои то веки, произошло что то хорошее.

Зачем этот пост? Думаю все понятно. Хотел бы услышать от вас что вы думаете о сабреддите, о моих действиях (вдруг претензии есть, кто знает), или вдруг у вас есть просто какие то вопросы/рекоммендации/идеи. За постом слежу, все комменты от юзеров в теневом бане будут зааппрувлены и прочитаны!

Спасибо за участие.

Hello. My whole homelab includes three machines right now, and also two VPSs that need monitoring. I need something that will be able to monitor docker container health, reachability of different endpoints, systemd units, and ideally monitor hardware usage. For hardware usage and alerting i can use Grafana and (forgot the second one, AlertManager?), but i have no idea about the others. Uptime Kuma is an option, but, in my opinion, it looks "too well" for monitoring lots of different metrics. Im looking for some slick dashboard that can help me collect everything in one place, but cant find anything like that. I mean, i can build myself a private solution, but im hoping that there are better options

Hello. My whole homelab includes three machines right now, and also two VPSs that need monitoring. I need something that will be able to monitor docker container health, reachability of different endpoints, systemd units, and ideally monitor hardware usage. For hardware usage and alerting i can use Grafana and (forgot the second one, AlertManager?), but i have no idea about the others. Uptime Kuma is an option, but, in my opinion, it looks "too well" for monitoring lots of different metrics. Im looking for some slick dashboard that can help me collect everything in one place, but cant find anything like that. I mean, i can build myself a private solution, but im hoping that there are better options

Hello. Two days ago, suddenly, networking in my docker compose started breaking down. The main thing affected was intercontainer communication, i.e. gitea and postgres couldnt reach each other while being in the same network (DNS resolved hostname - it couldn't connect). After, probably, 24 hours of attempts to fix this i rolled back to Docker 28.5.2. Containers could communicate with each other again! But then, i got a weird problem.

My current state - tailscaled and docker enabled and starting by default. All "exit node" functionality or --accept-routes are disabled for tailscale. docker has ipv6 enabled.

Services are reachable from tailnet, from localhost, but not from my LAN.

ss -tulpn confirms that services are listening, gitea confirms successful startup. I can open it using my domain and reverse proxy over tailscale.

Curl from machine in LAN when tailscale is disabled:

root@pve:~# curl 172.20.10.109:3070 curl: (56) Recv failure: Connection reset by peer

Curl from machine in LAN:

root@pve:~# curl 172.20.10.109:3070 -vv 16:51:52.198966 [0-0] * [SETUP] added 16:51:52.199034 [0-0] *   Trying 172.20.10.109:3070... 16:51:52.199142 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0 16:51:53.200220 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0 16:51:54.201363 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0 16:51:55.202484 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0 16:51:56.203581 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0

Iptables when docker and tailscale are running (cant copy rn, im sorry), top and bottom parts:

Top

Bottom

I tried experimenting. I disabled tailscale, restarted, ran conntrack -F - and suddenly everything became reachable from LAN again. But after plain restarts (without conntrack, both with tailscale disabled or enabled) - nothing changed, same results. I reinstalled everything - nothing changed.

I really need your help guys. Thank you for reading.