Anyone else exploring local AI as an alternative to commercial systems?
When 4o disappeared with no prior announcement last September, it was a shock and a huge disappointment. I'd been an avid 4o user for 6 months... not just for therapeutic purposes, but as a general creative brainstorming and research partner.
Even though 4o came right back at that time, the experience made it clear that depending on _any_ commercial AI system is a risk... someone else can turn it off or change it at any time with no warning.
I started exploring local LLMs, and while it was fun and educational, i didn't find anything that was nearly good enough to replace 4o.
But then a wave of new models come out recently, and some of them are really small and really powerful, outclassing models multiple times larger.
Recently, I put 20 local models through a series of tests to get a feeling for communication style, cultural knowledge, analytical ability, and appropriate responses to sensitive topics, including explicit trauma narratives.
Among those 20, the winners were clear: Gemma 4 and Qwen 3.6. I wouldn't say that they are 4o equivalents yet, but they are so much closer than models from a couple months ago. And as models get smaller, more and more people can run them on existing hardware. I've got one version of Gemma 4 that runs easily on my MacBook Air with only 16GB of memory.
I'm curious to know if anyone else is exploring in this area. I'd love to hear what kind of results you're getting! I'd also enjoy chatting with anyone who is interested in the topic.