Returned Studio M5 Max 64GB. Need more... something.
Ok first, this machine was absolutely amazing. I spent over a week testing every llm, inference engine, quant, bit, variant I could throw at it. Even built some automated testing, so that I could simply change the model, hit test and it would do a medium-difficulty coding task via prompt 6 times in a row, testing slightly different reasoning, and temperatures. That was great, my results were pleasantly surprising, and the speeds were more than I had hoped for.
The problem is I was in 90+% memory pressure the whole time, and I could still game (wow forever) while the LLMs were running somehow without stuttering or barely at all. All things considering, this machine is more than I could have hoped for, except I need more... More what though... I want next/next-flash models and you need 128 for that. My questions for that is do you still have enough resources left over to use the machine or is it headless at that point because its tight? Also, with the difference, do those models actually produce measurably better results?
My current thinking is that if the difference is neglibable, then I will go for the binned Ultra, which gives me more breathing room, and double memory bandwidth even though speed isnt the issue. The end goal was to collapse my m4 mini, and gaming desktop into that one machine to kind of do all the things it does now and more. I've been tinkering with blender, unity, various MCPs and I am well aware this would never replace cloud models and that isn't the goal. However, it could easily handle 50-70% of the work and leave planning and more difficult stuff to gpt/claude/grok.
Anyone else been through something similar, looking for opinions, thoughts. I dont make money from any of this and in the future if I did that would be great, but this is 1 part "want" and 1 part "potential to do....".