u/cryingneko

Hey everyone! It’s been a while, and I’m back with oMLX 0.4.5.dev1.
https://github.com/jundot/omlx/releases

I’ve been steadily committing changes since 0.4.4, but it was a little hard to decide where to cut the next dev release. I also wanted this release to include a meaningful attempt from the MLX kernel side, so it took a bit longer than usual. I hope you’ll understand.

The biggest change in this release is mainly relevant to people using an M3 Ultra, so apologies if this does not apply to your setup yet. - I’m also working on optimizing Gemma in a similar direction, so please stay tuned.

This release focuses on performance improvements for GLM-5.2, which I personally think is a big step forward for local AI, and MiniMax-M3, which has turned out to be a surprisingly useful model in practice.

Previously, these models “worked,” but honestly, I don’t think the long-context speed was where it needed to be for real use. With custom kernels, oMLX now gets a major speedup in long-context prefill. I also ran basic Needle in a Haystack tests and coding tests through Claude Code, and confirmed that quality did not collapse with the optimized path.

I hope this is a meaningful improvement for people using local LLMs in setups similar to mine.

Another major change is API-visible model profiles. You can now expose presets like 'qwen3-8b:thinking' or 'qwen3-8b:non-thinking' and call them directly through the API with the settings you want. Huge thanks to github pablomoralesm for this work: https://github.com/jundot/omlx/pull/1838

As always, this release was only possible because many people contributed their valuable time. I’m deeply grateful.

Thank you as well to everyone using oMLX, sharing feedback, reporting issues, and helping make the product better. It’s great to keep building local AI together!

Just shipped oMLX v0.4.2rc1: https://github.com/jundot/omlx/releases

If you've been running Qwen models on the VLM path, I owe you an apology. 0.4.0 had a tg slowdown caused by single-row decode falling into the slower batched cache path, which meant Qwen throughput on that path was noticeably worse than it should have been. If that's been your experience, I'm sorry for the trouble.

This release fixes it. Internal tg512 measurements show throughput recovered by about 1.48x, while Gemma performance stays stable. I'd strongly recommend upgrading if you're affected. I'll be testing the rc for about a day, and the 0.4.2 stable release should follow shortly after.

A few other things in this release while I'm here:

• Native MarkItDown document processing. Chat file uploads and the OpenAI API endpoint can now convert PDF, DOCX, PPTX, TXT, and Markdown inputs. You can also choose between MarkItDown conversion or VLM OCR for PDFs in the settings.
• Gemma 4 unified audio input. Gemma 4 unified models now accept audio alongside image inputs.
• Stability fixes across the cache, scheduler, and server.

Thanks to everyone who reported the regression and helped track it down. If you upgrade, I'd really appreciate hearing whether Qwen throughput is back to normal on your own setup, and any feedback on the new document conversion flow.

Hey everyone! oMLX v0.4.0 just landed.

This is the first official release of the new native Swift macOS app. The old PyObjC menubar app has been retired, and the macOS bundle now ships as a Swift app with a redesigned onboarding flow, settings UI, status surfaces, model management, and GitHub Releases based updater.

https://github.com/jundot/omlx/releases

The biggest user-facing change is the macOS app itself. First launch, server start/stop, model directory setup, downloads, update checks, menubar status, and the overall settings experience should now feel much more like a real Mac app.

Huge thanks to GitHub contributor popfido for the excellent work that drove the Swift transition. This is probably the biggest desktop-app change oMLX has shipped so far, and it substantially raises the quality of the app.

A few highlights:

• Native Swift / SwiftUI macOS app
• New onboarding flow
• Better menubar and server status behavior
• Standard Hugging Face cache model directory support
• Safer update flow with confirmation before download
• Memory guard tuning and CLI options
• More scheduler/cache stability fixes
• Guided grammar model setting
• Many admin UI, API, and model compatibility fixes

My long-term goal is still the same: I want oMLX to be "the app my friend who bought a MacBook yesterday can open and immediately try Local AI on." The Swift app is a big step in that direction.

At the same time, oMLX is still growing fast, and I know there are rough edges. If you try 0.4.0, I'd really appreciate feedback on the macOS app experience especially: first launch, model discovery, server start/stop, update checks, and anything that feels confusing or fragile.

Thank you again to everyone testing builds, reporting bugs, opening PRs, and giving feedback. The project has grown much bigger than I expected, and getting to improve it together with the open source community has been genuinely joyful.

What should come next? Feature suggestions, bug reports, and "this part is still confusing" feedback are all very welcome!

Hey everyone! v0.3.11 just landed. If you've ever run into stability issues with oMLX, I'd be really grateful if you gave this version a test.
https://github.com/jundot/omlx/releases/tag/v0.3.11

I'm well aware oMLX has had stability problems, and right now my number one goal is making it run reliably on low-memory Macs. The big change in this release is a full rewrite of the memory guard. The two confusing sliders are gone, replaced by a single Safe / Balanced / Aggressive dropdown, and oMLX now reads your live available memory and adapts in real time as other apps come and go.

On the structural side: A lot of oMLX's features currently rely on monkey-patching, so the project isn't as structurally stable as I'd like. The recent additions especially (MTP, DFlash, Deepseek v4 support) are in that fragile state, so they're getting a lot of my attention. To everyone who tests builds and reports bugs: thank you, genuinely. It makes a huge difference.

So, what should come next? I'll be honest about where my head is at. I want oMLX to be "the app my friend who bought a MacBook yesterday can open and immediately try Local AI on." So I'm a little cautious about features that are hard to use or hard to understand. My hope is that oMLX stays something anyone can pick up easily.

There are a lot of PRs waiting in the queue, and in the current structure I can't always bolt every feature on bug-free right away, but I'm always doing my best.

Thank you all for the constant support! I'm writing this partly as a thank-you and partly as shameless version promo (haha). I read every post here, even if I'm bad at replying (sorry about that), so if there's a feature you want, post it here on the subreddit, open a GitHub issue, wherever works for you. I'll always see it.