4 agents, mixed routing: some cloud, some local

Been experimenting with per-request privacy routing in Trooper. Wanted to see if it actually works when you need some requests to stay local but don't want to give up Claude for everything else.

Ran 4 agents. Two asked about public stuff (OAuth vulnerabilities, Redis vs Memcached). Two handled internal data (API keys, customer names).

Agent 1 - Claude:

"Top 3 OAuth2 vulnerabilities?"
Public knowledge, let Claude handle it.

Agent 2 - Qwen (local):

"Format this: api_key=sk-prod-xxxx, vault_url=https://vault.acme.io"
Has credentials. Stays on my machine.

Agent 3 - Claude:

"Redis or Memcached for sessions?"
General question, use cloud.

Agent 4 - Qwen (local):

"Summarize: 47 tickets. 3 had PII (Alice Johnson, Bob Chen, Maria Garcia)"
Customer names. Can't send that to Anthropic.

Everything worked. Cloud agents took 2-4 seconds. Local ones were faster (1-2s). The credentials and customer names never hit the network.

Why bother

I don't want my entire coding session local. Qwen is good but Claude is better for complex stuff.

I just want specific messages to stay on my hardware when they contain:

• Internal service URLs
• API keys or tokens
• Customer data
• Anything I wouldn't put in a blog post

The per-request control is the point. Not "all local" or "all cloud" — mix them based on what you're asking.

How it's different from my last post

Last time I showed what happens when Claude quota runs out. Trooper falls back to Ollama automatically.

This is proactive. You tell it "keep this one local" before sending. Different problem.

Both use the same context system so the local model knows what happened in the cloud part of the conversation.

What doesn't work great

Qwen isn't Claude. It's fast and fine for formatting/parsing/summarization. But if you need deep reasoning, route to Claude.

You need Ollama running. I use qwen2.5:3b (2GB, fast enough) or 7b if I want better quality.

Repo: https://github.com/shouvik12/trooper

Still iterating on this. Let me know if you hit edge cases or have ideas for better routing heuristics.

Trooper started as a proxy for Claude API calls. When quota hits, it falls back to local Ollama with context preserved.

Today it handles real human conversations — and I shipped a feature worth talking about.

The scenario:

You're mid-conversation with Claude through Trooper. Architecture decisions, authentication design, system planning. Everything going to the cloud.

Then a sensitive detail comes up. An internal service URL. A proprietary system. Something you'd rather not send outside your machine.

Previously your only options were stop the conversation or send it anyway.

Now there's a third:

"x_force_local": true

One field in the request body. That specific message routes to local Ollama with full session context intact. Next message goes back to Claude automatically. No restart. No lost context. No interruption.

Left terminal — a real conversation. Four turns. Claude handles architecture and auth questions. Developer types local. Sensitive vault URL stays on machine — Ollama answers it locally. Claude summarises after.

Right terminal — Trooper routing decisions in real time:

🔒 Developer requested local-only (x_force_local) — skipping cloud
🔒 Local: ollama (force_local) | privacy mode | session saved: 18 tokens

What actually happened with the vault detail:

The raw vault URL never left the machine - Ollama handled it locally. Trooper's SITREP then created a compressed abstraction of the session state. Claude received that abstraction, not the raw message.

What's different here vs LiteLLM or Bifrost:

The individual pieces exist elsewhere — local routing, fallback, context compression all appear in fragments across different tools. What's different is the composition:

A session-stateful LLM router that enables per-turn execution locality with a shared compressed memory layer enabling cross-provider continuity.

The differentiator: execution locality is a runtime decision inside a persistent conversation state machine.

They route between clouds. Trooper routes to your machine. Per message. Mid-conversation. Without breaking anything .

Three reasons to use x_force_local:

• Privacy — sensitive payload never leaves the machine
• Cost control — force expensive turns to local
• Offline mode — keep working when cloud is unavailable

How context is preserved:

Trooper uses a 3-layer memory system across provider switches:

• Anchor — first 2 turns, always preserved verbatim
• SITREP — compressed abstraction of middle turns
• Tail — recent turns within token budget

The local model always knows where the conversation was — without receiving raw history from the cloud session.

Repo: https://github.com/shouvik12/trooper

Most people use Ollama as a primary local model.
I ended up using it differently — as a continuation layer when the cloud fails.

Here's how a real session played out:

Turn 1 - Claude failed (credit_balance)
Trooper detected the error, fell back to Ollama, and carried full context:

X-Trooper-Decision: ollama (fallback: credit_balance)
X-Trooper-Summary: claude → ollama (credit_balance) | context ✓
X-Trooper-Session-Saved: 12 tokens

Turns 2–6 — simple queries (local only)
Rule-based classifier detected simple turns. Ollama handled all of them directly.
Cloud was never contacted again.

X-Trooper-Decision: ollama (simple turn) | cloud skipped
X-Trooper-Session-Saved: 76 tokens

Ollama handled 5 out of 6 turns in this session.

The key problem with fallback

Local fallback usually fails because the model starts cold — no context.

What fixes it

Before sending to Ollama, Trooper compacts the session into a structured SITREP:

{
  "intent": "building a go proxy",
  "stage": "in_progress",
  "open_loops": ["streaming pending"],
  "recent_actions": ["deploy monday"],
  "confidence": 1.00
}

• extracted rule-based
• no cloud LLM call
• no added latency

So Ollama doesn’t restart the conversation — it continues it.

What this turns Ollama into

• Reliability layer → absorbs cloud failures
• Execution layer → handles simple prompts locally
• Cost layer → avoids unnecessary API calls

Not just a local alternative — a fallback infrastructure layer.

There’s been some early organic pull on this:

379 clones, 166 unique cloners, 1,319 views, 196 visitors in ~14 days
No launch post — just devs finding it and trying it.

What Trooper is

A drop-in proxy. Zero dependencies. Pure Go.

Your app → Trooper → Claude
                   → fallback → Ollama
                   → continues seamlessly

Curious if others here are using Ollama this way — as fallback infra rather than primary?

→ https://github.com/shouvik12/trooper

Shipped v3.0 today based on feedback from the thread yesterday.

Three things added:

• Circuit breaker — if a provider fails 3 times in 60s, Trooper skips it automatically. No more wasted round trips hitting a known-dead provider on every request.
• Observability log lines — every request now surfaces what happened clearly in the terminal
• X-Trooper-Summary header — one line on every response showing exactly what Trooper did

Still zero dependencies, single Go binary.

github.com/shouvik12/trooper

Built Trooper to solve a problem I kept hitting — Claude's API quota running out mid-conversation and breaking my app flow.

I built it in an evening, using Claude as my coding assistant throughout. Claude helped with the Go proxy architecture, the fallback logic, and debugging the streaming response handling.

What Trooper does:

• Sits between your app and Claude's API
• When Claude returns a 429 or 402, silently reroutes to a local Ollama model
• Preserves full conversation history across the switch
• Zero code changes in your app — just point your base URL at localhost:3000
• Streaming support
• Configurable fallback trigger codes
• 401 errors surface properly — bad keys never masked

Getting good traction from the builder community which has been encouraging.

It's free and open source.

GitHub: github.com/shouvik12/trooper

Happy to answer questions or take feedback from the Claude community.