Ollama 0.31: Faster Gemma 4 on Apple Silicon with MTP. Here is my test showing a 56% boost on M1 Pro 16GB (2021)
Gemma 4 models are up to 90% faster with Ollama 0.31 on Apple Silicon. The speedup comes from multi-token prediction (MTP). This was achieved through contributions to the MLX kernel and the improvement isn't limited to Gemma 4 models. Gemma 4 is the first model to receive this improvement.
Note: You need to download new mlx tags to get this improvement.
Scope
Ollama's official benchmark measures Gemma 4 12B on the Aider polyglot benchmark using an M5 Max.
I wanted to run a simpler benchmark on an older device. I tested gemma4:e4b on a single JSON generation prompt. This means a different model size and a different workload. This isn't a validation or refutation of their numbers, just a separate data point on smaller hardware with a narrower task.
What I Tested
I compared gemma4:e4b and gemma4:e4b-mlx on M1 Pro MacBook Pro 16GB with a simple JSON array generation prompt. I picked this prompt because MTP especially performs better when output predictable (closing brackets, repeated identifiers, boilerplate).
Generation Speed (Higher is Better)
Prompt: "Generate a JSON array of 30 fake user objects with id, name, email, and signup_date fields."
| Model | tokens/second |
|---|---|
| gemma4:e4b | 32.60 |
| gemma4:e4b-mlx | 50.95 |
It performs 56% better with mlx on a 5 years old machine.
Conclusion
This is a meaningful improvement but because it comes from multi-token prediction, it mostly benefits predictable generations. That is, it won't give the same performance improvement for all prompts.
The app I used: Reins: Chat for Ollama
Read More: The Blog Post