▲ 35 r/StrixHalo
Gufo engine seems a good open source choice vs. the closed Halogen
Recently tried running the Gufo engine (GitHub - gufo-org/gufo: Strix Halo inference engine. Qwen Flash Next Q4_K_XL: 1,700.52pp, 60.39tg single user, 162.98 tok/s 8 users; Qwen27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2 · GitHub) on my Strix Halo device (GPD Win5 128G).
It's performance is close to the Halogen (much better than llama.cpp variants on the prefill performance). I'm switching to this as daily use.
Also created a short video as introduction on this: https://youtu.be/r-pFZzIMvgU
u/zweily — 5 days ago