u/boxwrenchx

I never expected to run a 321B model at home, so I jumped on this as soon as I saw it last night.
Project Maya:

(github.com/mw00/project-maya) runs GLM-5.3-Flash by keeping the hot experts in VRAM, the next ones in RAM, and the rest on NVMe. It was NVIDIA-only. So I ported it to Linux/ROCm, and the author merged it into v1.0.11 as experimental AMD support.

Maya-S quant, 8K context, ROCm 7.2, 4K-token prompts, greedy:

| Hardware | Prefill | Decode |

| RX 7900 XT 20 GB | ~415 tok/s | ~15 tok/s |

| Radeon AI PRO R9700 32 GB | ~500 tok/s | ~20 tok/s |

| Strix Halo (Ryzen AI Max+ 395, 8060S, 128 GB) | ~210 tok/s | ~18 tok/s |

| R9700 + 7900 XT (layer split + MTP drafting) | ~490 tok/s | ~34 tok/s |

The dGPU box has 192 GB of RAM, so most experts live in VRAM or pinned RAM.

With less RAM, more comes from the SSD and decode drops.

AI-assisted, openly

Just like everyone said, Ox Alpha is not a Google model. Also in case you need to hear it, Google abandoned 3.5 pro. It doesn't mean they're done, it doesn't mean they'll never again release a good model but it does mean it's silly to ignore all evidence otherwise.