Login

u/ProgramOver9309

r/oMLX r/LocalLLM r/hermesagent r/LLM
▲ 7 r/oMLX+1 crossposts

MacBook Pro M5 Pro 48GB Ram

I have just purchased a MacBook Pro M5 Pro with 48GB RAM. I was wondering if any out there have the same RAM and what models they are using locally that fits with enough headroom. What inference engine, and are you using MLX or GGUF? Im only going to use it for some light terminal work, some automation, and some python and light html work. Probably going to use Hermes as harness. Please share your setup or recommondations for 48 GB Ram on the MacBook Pro M5 Pro. Thank you!

reddit.com
u/ProgramOver9309 — 3 days ago
▲ 105 r/hermesagent

Rate my setup 1-10

Im running hermes ( Agnes Ai ) for free on Blackberry Passport through my Raspberry Pi 5 to get these 2 bad boys up and running. Ready to go Local Ai all the way!

u/ProgramOver9309 — 23 days ago
▲ 1 r/hermesagent

Stuck in a loop

Could somebody help me please who knows how to prevent hermes from getting stuck in a loop? Im running Qwen 3.6 27B Q8 on a Asus Ascent GX10 GB10 with llama.cpp. These are my flags:

llama.cpp/build/bin/llama-server -b 4096 -m /home/spark/models/Qwen3.6-27B-MTP-Q8/Qwen3.6-27B-Q8_0.gguf --host 0.0.0.0
--port 8000
--flash-attn on
--ctx-size 131072
-ngl 99
--temp 0.6
--top-p 0.95
--top-k 20
--presence-penalty 0.0
--repeat-penalty 1.0
--spec-type draft-mtp
--spec-draft-n-max 2

u/ProgramOver9309 — 1 month ago
▲ 1 r/LLM

I run an ASUS GX10 GB10 with 128GB unified memory — basically a DGX Spark. After trying a bunch of local LLMs, I landed on Qwen3.6-27B Q8 Dense. Yeah, MOE would be faster on this hardware. I still prefer quality over throughput.

7-8 tok/s is too slow for coding though. That's what I'm getting right now. Anyone have speculative decoding working on a similar setup? dflash, pflash, whatever — I need at least 20 tok/s to use this for actual coding work.

reddit.com
u/ProgramOver9309 — 2 months ago