u/USBhost

Hello lads I just wanted to provide my experiences I have with MiniMax-M3 MXFP4_MOE under llama.cpp with the PR. I used MiniMax-M2.7 before at a higher quant and it's defiantly better than qwen3.6 26B, it was able to find coding bugs that the 26b just did not see.

My system is:
AMD EPYC 7F72 24-Core
256GB 8x32GB of 3200R DDR4 RAM
RTX A6000 48GB GPU
Samsung 990 EVO 4TB

It seems to work fine but my PP is so bad lol, it's like only 22tk/s. I mean if you have the whole day you could use it for stuff.

So the MXFP4_MOE quant of M3 is 256GB, the exact size my system ram has. And of course you lose some with the OS. So I was placing my bets on mmap only pulling what it needs. In my case around 6GB was used on my Poxmox/OS server (with nothing running). No SWAP

So I just ran the normal -ngl 99 and -cmoe, What I saw for around 5min on the first prompt It was maxing out my nvme. But after whatever it needed to reread it never had to do it again. My Prompt processing was 22tk/s and the inference was almost 12 and it got slower as the more context you used. It was still around 11+ at 10k context. According to llama.cpp I had the max context of like 280k but idk how bad inference would be at that size lol

In my conclusion it's usable for me but for stuff on the side. Anyone else with similar setups as me?

Things that would be fun to see if I upgraded my cpu to one that has 8 CCDs vs just the 6, would things be 25% faster.