How to with a 128gb evo-x2, MacBook Pro 48gb & M5 Pro, and 4080 super (16gb) desktop.
What’s the best way to utilize this setup?
I want to develop local llm products for SMB, learn AI, and build a great memory system. I really need a “life AI” also, that handles family, health, hobbies, career, etc. in addition to dev & startup work.
Ultimately, I’d like a local LLM to be my daily driver for everything and escalate to frontier models to only learn new things.
I’ve mad ADHD and rapid fire shit to AI and get it more sidetracked than a human in the same situation. I’ve got a decent memory layer now that’s handling this pretty well. but am I reinventing the wheel?
Currently I’m using code as a front end for all of these things. With some internal orchestrators, and leads and an oracle that knows all - but never acts, only directs other orchestrators and leads.
The 128gb evo-x2 is running qwen 35b MoE. About 500-900tok/s PP and ~50tok/s decode. All of its work is focused on business products not as a daily driver currently.
Curious what others are doing, and what everyone’s thoughts are…