Login

u/FootballSuperb664

r/LocalLLM r/ClaudeAI r/MacStudio
Run Claude Code against a local Gemma 4 or Qwen 3.6 - no API key, no cost, works on any Apple Silicon Mac
▲ 140 r/MacStudio+2 crossposts

Run Claude Code against a local Gemma 4 or Qwen 3.6 - no API key, no cost, works on any Apple Silicon Mac

If you have an Apple Silicon Mac you can run Claude Code completely locally (and free) by pointing it at a local server. Here's how:

Setup (2 minutes) brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve brew install --cask mlx-core # GUI menu bar app brew install mlx-serve # CLI server only mlx-serve run gemma-4-e4b-it # downloads + starts the server (not needed if you use GUI)

Then launch Claude Code with:

ANTHROPIC_BASE_URL=http://localhost:11434 \
ANTHROPIC_API_KEY=local \
ANTHROPIC_DEFAULT_MODEL=mlx-serve \
claude

That's it. Claude Code streams, tool calls, thinking blocks, multi-turn - all work against the local model via the Anthropic Messages API.

What runs well locally

  • Gemma 4 E4B 4-bit (recommended starting point, ~105 tok/s decode on M4 Max)
  • Qwen 3.6 27B 4-bit with native MTP spec-decode (~36 tok/s, 1.43x faster on code tasks)
  • Qwen 3.5 4B/9B for faster iteration cycles

Full walkthrough + tips for which models work best for coding tasks: https://mlxserve.com/claude-code-local/

The server is mlx-serve - MIT, no Python required, single binary. brew install mlx-serve

GitHub: https://github.com/ddalcu/mlx-serve

u/FootballSuperb664 — 3 days ago
▲ 129 r/MacStudio

Your Mac Studio can run DeepSeek V4 Flash (284B) fully local now, native Zig server, zero Python

I've been building a thing for Apple Silicon and I think r/MacStudio is exactly the crowd it's for.

Only makes sense if you have a lot of unified memory, which is basically the whole point of this machine.

It runs DeepSeek V4 Flash, the 284B-parameter model, fully local on a 96GB+ Mac.

It's called mlx-serve, a native Zig inference server (no Python) plus a little macOS menu-bar app called MLX Core. Some things that might matter to you:

- Faster than LM Studio (MLX) on every model I've benchmarked, identical 4-bit weights, same machine, same prompts. Tested  across Gemma 4 (E2B/E4B/31B/26B-A4B MoE) and Qwen 3.6 (27B/35B-A3B).

- Qwen 3.6 support (incl. the 35B-A3B MoE), Gemma 4, Llama, Mistral, Nemotron-H, LFM2.5, and more.

- Speculative decoding — Prompt Lookup Decoding (works on any model, up to ~1.6× on echo/RAG) and a Gemma 4 drafter (+~30% on code). Exact same output, just faster.

  - OpenAI and Anthropic APIs — so you can literally point Claude Code at your Mac Studio and run it against a local model.

  - Continuous batching (~1.6× at 4-way parallel), KV-cache quantization (~4× smaller bigger context / more parallel requests),  built-in agent + MCP tools.

  - Free & Open Source, MIT licensed.

ddalcu.github.io
u/FootballSuperb664 — 1 month ago
▲ 11 r/LocalLLM

MLX-serve vs LM Studio on Apple Silicon ~40% faster in my benchmarks (w/ MTP/PLD)

Benchmarked mlx-serve against LM Studio on Apple Silicon today, roughly +40% faster overall depending on types of workload when using new Gemma4 drafter MTP and PLD in other models.

The gap is widest on echo/repetitive tasks like agentic code editing where speculative decoding really kicks in (+122% on Gemma 4 E2B echo), and more modest on free-form generation (~+20%). Both using the same MLX weights over HTTP so it's a pretty apples-to-apples comparison.

It's a native Zig server so no Python in the stack, and it exposes OpenAI + Anthropic-compatible APIs if that matters to your setup. Posting in case anyone else is trying to squeeze more out of their M-series chip.

https://github.com/ddalcu/mlx-serve

u/FootballSuperb664 — 2 months ago
▲ 2 r/LocalLLM

I see a lot of model quality benchmarks, but none that test the actual endpoints of servers to make sure they work well.

If we build agents locally, how do we know LMStudio/Ollama/MLX work properly ?

Talking about proper spec testing on: Responses API, Chat Completions API, Anthropic Messages API.

Found this repo, but it's only for Responses, is there one for Completions and Messages ?

https://github.com/openresponses/openresponses

I see a lot of problems, and crashes when you go beyond simple Chat Completions, LM Studio specifically.

u/FootballSuperb664 — 2 months ago