Building Conifer, an open-source local inference runtime, launching June 1st (free + open source)
Hey all, I hope you're doing well! I'm building an open-source runtime called Conifer (conifer.build) and wanted to share it with this community.
The idea is that most of the pain of running models locally isn't the model itself, it's everything around it: setup, storage, quantization, memory, and scheduling work across whatever hardware you've got. Conifer handles that layer so local inference runs fast and stays private, instead of being a fragile fallback to the cloud.
We've got funding to build it, and right now we're already beating llama.cpp on some benchmarks, with more improvements coming as we prepare for launch.
We're launching our beta on June 1st for our waitlist users. It'll be completely free and open source, so you'll be able to install it, point it at a model, and see what it does for yourself. If you're interested, we'd love for you to join the waitlist at http://conifer.build/feedback and check it out.
Happy to answer any questions in the comments!