u/EricBuehler

mistral․rs provides web search and safe, sandboxed code execution functionality to allow you to build powerful agentic apps with Gemma 4 12B.

There's also full multimodal support, so you can build with audio, image, and video.

Installation is one-step:

# Linux/Mac  
  
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.sh | sh  
  
# Windows  
  
irm https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.ps1 | iex  

Then, just run:

mistralrs run --agent -m google/gemma-4-12B-it --quant 4

This will launch an OpenAI and Anthropic-compatible HTTP server, with a built-in UI web chat at localhost:1234/ui.

You can also use MTP:

mistralrs run --agent -m google/gemma-4-12B-it --quant 4 --mtp-model google/gemma-4-12B-it-assistant

Check out the GitHub for more details: https://github.com/EricLBuehler/mistral.rs
Documentation: https://ericlbuehler.github.io/mistral.rs/