All local models suck. Even DeepseekV4 can only handle instructions. Prove me wrong plssss
I can spend $25k-$100k on a local computer, call it a business expense. I'll spare the details.
But no... These models just suck.
I've tried every single model outside the 1.6T DeepSeekv4. Maybe if people think that is useful, I'll do a trial Vast.ai server.
I feel like I'm letting my ultra privacy focused customers down. I've been trying for 1+ months and probably spent $400+ on vast.ai server tryouts.
I know some people are getting weather and stonk prices with crappy 35B models... We have significantly more complex stuff. Combining 260 page pdf docs with a massive dropbox.
Only Opus has worked.
Maybe I need to lower expectations? Maybe I need to have Opus make MCP/CLI-like skills for ~500B models?