u/horratiocornbl0wer

Hey r/StrixHalo — I built hal0.dev with the goal of optimizing for exactly this hardware and extracting the best possible performance, functionality, and value from it.

We're finally ready and opening public beta this weekend. Would love to have you kick the tires — I've had limited testers so far and we're ready for more.

The idea. A Strix Halo box is a genuinely special piece of kit — Radeon iGPU, XDNA NPU, and one big unified-memory pool — and hal0's goal is to extract the most performance, value, and functionality possible from it.
Chat, embeddings, rerank, transcription, live speech, image gen — answers on one local /v1/ API.

This is my first real shot at something this ambitious, so the philosophy is deliberately narrow: high impact features, reliable, proven tools, wired up automatically, and integrated deeply across the platform.

One-line install builds and wires up — automatically

• Models across llama.cpp (Vulkan/ROCm FPX / MTP) and the XDNA NPU via FastFlowLM — running co-resident, highly tuned - chat, embed, rerank, vision, STT, TTS, and image gen via ComfyUI
• Hermes agent provisioned with auto model/slot detection and custom Hindsight memory integration with MCP access for outside agents/tools - no manual config
• Operator Board — a multi agent capable Hermes-backed Kanban that tracks tasks across profiles, lanes, and projects, with gated actions pausing for your sign-off and live agent chat beside it to help you orchestrate.
• Open WebUI for chat, RAG, and more, alongside the dashboard - models & slots appear automatically.
• Custom Hindsight memory + knowledge graph (NPU Extraction by default) wired to Hermes out of the box and exposed via MCP for Claude, Pi etc.
• MCP server exposing hal0 admin surfaces to agents — keeps agents in the know about the entire lab structure and lets them tweak it on your command.

Slots: every model runs in a "slot" — one model, one container, with a typed lifecycle and a GPU arbiter that assigns unified-memory to either always-on concurrent LLMs or image gen, one group at a time — so GPU workloads never fight over the pool, yet multiple LLMs stay concurrent and always ready.

Agents & memory: striving for the deepest, most seamless Hermes integration possible — kanban, delegation, and hal0 administration, all out of the box. Memory is a constantly improving shared brain: a fully built-out Hindsight custom-provider system with a primary private bank (seeded per child profile) plus a shared bank with MCP access, so agents like claude-code, pi, and opencode can learn from and teach your agent as the homelab evolves.

Developed on a Ryzen AI Max+ 395 / 128 GB. I run mine in a Proxmox LXC for the exceptional quality-of-life wins — resource sharing/allocation without being captured, plus the reliability. Bare-metal Ubuntu and WSL2 (WIP) paths are in the docs too. It's hardware-agnostic in principle but tuned for Strix Halo first, particularly on Proxmox — NVIDIA/CUDA is being worked toward as a supported runtime device, but don't count on it working just yet.

Open-source, Apache-2.0. Come kick it around and tell me what falls off 🙂

⭐ https://github.com/Hal0ai/hal0 - Give Us A Star!

🗪 https://discord.gg/n2ftGqYr8 - Join Us In The Discord!

💫 https://hal0.dev - Promo, Info, Docs & More

https://store.minisforum.com/products/minisforum-deg2-oculink-egpu-dock

Took me almost a month to get from them.. And I had other plans by time I did. Anyone interested in getting it 1. Quick and 2. Discounted?

I'll take photos for serious buyers.

Thansk!

hey r/strix_halo,

5/23/26 - MAKING SOME BIG CHANGES - CHECK BACK IN TONIGHT.

- site + docs: https://hal0.dev - Updated with the next version features added.
- source: https://github.com/Hal0ai/hal0 - Look for v0.3 Release Soon - Don't bother with V0.2

A little over a month ago I picked up a minisforum MS-S1 max and have been lurking in the forums, this community and others since.
Getting this box has been an incredible experience in my developer/ai journey and I've learned an insane amount by tinkering with it, building out my homelab on it. etc.
i've been head-down for the last week building a new and hopefully open source worthy version of hal0ai, an open-source homelab AI inference platform aimed squarely at the linux box you already have in the rack. v0.2.0-alpha just shipped and i'd love some extra hands kicking the tires before i call anything stable.

tl;dr what it is

hal0 is the Proxmox friendly, OpenAI-compatible inference layer + dashboard and agent environment i wished existed whenIi first plugged in my ms-s1 strix halo box into the home-rack.
Systemd-managed model "slots" with a real lifecycle state machine, prewired OpenWebUI chat tab, cosign-signed self-update with rollback, no telemetry. The slot's are easily spun up/down, added/removed and models subbed in/out.

happy in a privileged proxmox LXC with full iGPU + NPU passthrough, behind your traefik.

heads-up

this is the first open-source project i've shipped at this scale, so… be gentle. there are rough edges still i'm sure i'm setting up a discord and git issue tracker. APIs may still shift before v1.0 — don't pin anything load-bearing to the alpha surface yet.

links

- site + docs: https://hal0.dev

- source: https://github.com/Hal0ai/hal0

- discord (7-day invite, i'll refresh weekly in the repo): https://discord.gg/uTYRn54A

 

thanks for reading — tear it apart.
Alexander T

 

Hi all, about 2 months ago I picked up the MS-S1 MAX and have been having an amazing time adapting my skillset from a web developer to tinkering AI lunatic. I've been custom building an inference platform with Proxmox running an LXC w/ a custom setup that manages "slots" each running a type of strix-halo-toolboxs (vulkan, ROCm, and giving vllm a try now also) - It's been working great, models always hot, no pipeline backup/switching models and I can have Hermes agent running swarms/utilities/coding/ while the main chat stays responsive... so... Next up....

Looking for real-world experience from anyone who's tried to make an AMD iGPU and an NVIDIA dGPU work together for LLM inference, specifically over Thunderbolt 5 / USB4 v2. Not asking "should I do this" — I've already got the hardware. Asking "what's the cleanest path that actually works today."

Hardware:

• Minisforum MS-S1 MAX — Ryzen AI Max+ 395 (Strix Halo), 128GB LPDDR5x-8000 unified memory. iGPU is the Radeon 8060S (40 RDNA 3.5 CUs). Currently running Proxmox. UMA frame buffer cranked up so the iGPU addresses most of system RAM as VRAM. Ports include 2× USB4 v2 (80 Gbps, TB5-compatible) and 2× USB4 v1 (40 Gbps).
• RTX 4080 — 16GB GDDR6X. Currently in a separate Windows box, about to relocate.
• Minisforum DEG2 — eGPU dock with both OCuLink (64 Gbps PCIe 4.0 x4 direct) and Thunderbolt 5 (80 Gbps) modes via a physical toggle. Plan is to run it in TB5 mode into the MS-S1's USB4 v2 port for one-cable docking. (Aware that TB5's internal PCIe tunnel is still effectively PCIe 4.0 x4 ≈ 64 Gbps for GPU traffic — so similar raw bandwidth to OCuLink with a small latency penalty for the protocol translation. Open to flipping to OCuLink mode if anyone has data showing it matters for inference workloads.)
• Plan: pass the 4080 through to a Linux VM (CachyOS or Fedora) on Proxmox. Use the iGPU for the host console and AI workloads. Stream the desktop via Sunshine/Moonlight when needed.

What I want to do: Run 70B+ models (Llama 3.3 70B, Qwen 2.5 72B, GPT-OSS 120B, DeepSeek-V3, etc.) and ideally use both GPUs cooperatively rather than picking one. The iGPU has the memory capacity (128GB unified pool, ~256 GB/s bandwidth) but the 4080 has the compute and the fast GDDR6X (~700 GB/s, 16GB). They feel complementary on paper.

What I've already looked into:

1. llama.cpp Vulkan backend with --tensor-split — both GPUs are Vulkan-capable, so in theory I can layer-split a 70B across them. The 4080 takes a perf hit running Vulkan instead of CUDA, but I gain its 16GB of fast memory in the pool. Anyone actually benchmarked this on a Strix Halo + NVIDIA combo over TB5? Real TPS numbers welcome.
2. llama.cpp --rpc mode — looks like the cleanest way to get CUDA-native on the 4080 and ROCm/Vulkan on the iGPU in the same logical model. Each device runs its own engine, they coordinate over RPC. Anyone running this in production? How fragile is it?
3. Speculative decoding — small draft model on the 4080 (e.g., Llama 3.2 3B in CUDA), big target model on the iGPU. Feels like it might be the highest practical win. Has anyone wired this up across mixed-vendor devices?
4. KTransformers — purpose-built for "consumer GPU + lots of system RAM" MoE inference (DeepSeek-V3 etc.). Does it actually play nicely with Strix Halo's iGPU treated as the "fast system RAM" tier, or does it want a real CPU+dGPU split with no iGPU in the loop?
5. MoE expert offloading — pin hot experts to 4080 VRAM, cold experts in iGPU unified memory. Aphrodite/ExLlamaV2 territory. Anyone doing this with Mixtral 8x22B or DeepSeek-V3 on similar hardware?

Specific questions for the community:

• TB5 vs OCuLink for inference workloads: Both are effectively PCIe 4.0 x4 for the GPU traffic. Has anyone benchmarked a meaningful difference on real inference (not synthetic transfer tests)? My read is layer/pipeline parallelism is fine on either, but tensor parallelism would suffer from all-reduce traffic regardless. True?
• Is there any inference engine in 2026 that natively handles heterogeneous CUDA+ROCm in the same process, or is RPC / separate-engines still the only path?
• Any gotchas with NVIDIA GPU passthrough on Proxmox over Thunderbolt 5 specifically? IOMMU groups behaving themselves on Strix Halo when the dGPU is hot-pluggable in theory? I'd planned to lock the 4080 to vfio-pci at boot to keep things deterministic.
• Anyone running into issues with the Strix Halo iGPU console + NVIDIA dGPU passthrough coexisting? The "disable IOMMU/VT-d to avoid black screen" warnings I see in the eGPU community directly conflict with what I need for vfio passthrough.
• For someone who'll be tinkering with this, what's the order of operations you'd actually recommend — Vulkan tensor-split first to validate, then RPC, then KTransformers? Or different?

Happy to share back benchmarks once it's up. Thanks in advance to anyone who's been down this road.