r/Applesilicon

For anyone who hasn't seen it, Darkbloom is a network where Macs run open models (Gemma, Qwen, etc.) for people paying for API access. Your Mac loads a model into unified memory and answers requests when they get routed to you.

Wanted out why some Macs get tons of work and others get basically none. Some stuff I found:

• in their public stats from late Sept (~1,140 Macs), the 96GB+ machines were under a third of the network but did 60% of the tokens. 32GB and under was 18% of Macs and did about 1%
• requests go to whoever can answer fastest, so bandwidth matters too. Max/Ultra chips have a real edge
• 32GB is pretty much the floor. Gemma 4 26B alone takes ~24GB. At 64GB everything fits
• there are way more Macs than requests right now (under 10% of capacity actually used), so a lot of the time nobody's getting much
• lid closed or asleep = offline, which caught me out a couple times

I ended up building an app (and a website bc why not) to track and control all this (which model is earning, idle time, why it's not getting work, etc.): bloomgauge.io

The core is open source (MIT): github.com/cookder/bloomgauge

Anyone here running it on an Ultra or 128GB+? Curious how your numbers compare. Also interested in M5 versus everything else