Hey everyone,

I just open-sourced TuneForge.

The goal is simple: let your coding agent manage the full LLM improvement loop without ever leaving the chat window.

You can now tell your agent something like:

“Build me a customer support bot from this FAQ”

…and it can:

• Generate a clean synthetic instruction dataset (with LLM judging for quality)

• Run LoRA supervised fine-tuning on any Hugging Face causal LM

• Do a quick policy-gradient RL step using Ollama as the reward judge

• Merge the adapter, evaluate on a test set, and iterate

Everything runs locally, uses 4-bit quantization so it fits on modest hardware, and uses background jobs (with job_id polling) so long training tasks don’t freeze the MCP connection.

It’s built around the Model Context Protocol (MCP) for seamless integration with Claude Desktop, Cursor, Zed, Continue.dev, etc.

Tech: Python + Transformers + PEFT + bitsandbytes + Ollama + SQLite for job state.

Super early stage (just released), MIT licensed.

Would love feedback or ideas on what to add next. If you’re into agentic fine-tuning workflows, give it a try and let me know how it goes!

I've been building an open-source agent (Apache-2.0) and wanted to share it here because it's designed to be fully local and self-hostable: it talks to any OpenAI-compatible endpoint, so Ollama / llama.cpp / vLLM / LM Studio all work as the backend. No cloud lock-in, your keys and data stay yours.

The core idea is LLM-Fusion: for the hard steps it can run a panel of models on the same prompt, have a judge model cross-check them (consensus / contradictions / blind spots), and a synthesizer write the final answer. Locally this is fun because you can mix a few small local models and let them cross-check each other. A cost/latency-aware router keeps easy turns on a single model so you're not paying panel latency for everything.

Beyond that it's a full agent: plan -> act -> verify-or-revert (it runs your tests and treats the result as ground truth), layered memory (SQLite + FTS recall, cross-session profile, consolidation), a governance kernel, cron/proactive jobs, MCP client + OpenAPI-to-tool import, and an isolated subagent/crew layer (parallel git worktrees with per-worker verify gates). Runs on a laptop or a $5 VPS via Docker.

Honest status: it's alpha - 463 tests, mypy --strict clean, but no production mileage yet. Local reasoning quality obviously depends on the models you point it at, so I'd genuinely love to hear which local models people find good enough to actually drive an agent loop (reliable tool use + self-correction) - that's the make-or-break for going fully local.

Repo: https://github.com/brcampidelli/chimera-agent

Un IDE dove il codice lo scrive l'AI, lo lanci tu, e il sandbox fa il resto.

Si chiama WebCraft. È dentro NHA 3rdArm gratis.

A parte questo, cerco disperatamente community per portare avanti il progetto! Tra lavoro e impegni, sta diventando difficile......siete interessati? L'applicativo ha tante alte features, tra cui una sezione avanzata per i connettori con market place

👉 nothumanallowed.com

https://nothumanallowed.com/3rdarm

I ran the same market-entry brief through three different AI models. The result was revealing.

I asked three models to independently create a client-ready market-entry brief for launching a privacy-first AI personal assistant for small businesses in the UK.

The models were:

1. Claude Fable 5 via Claude Subscription
2. GPT-5.5 via ChatGPT/Codex
3. qwen3.6:27b running locally via Ollama

Each got the exact same task. They could use web research. They could not see each other’s answers.

The brief was for a product that is local-first, helps with email, calendar, documents, reminders, research, and workflow automation, and positions itself around privacy, local storage, user control, and optional cloud model access.

The target market was UK small businesses, freelancers, consultants, and agencies.

The output needed to include segmentation, customer pains, competitor landscape, positioning, pricing, go-to-market strategy, risks, a 90-day launch plan, and a clear recommendation on whether the company should pursue the market.

Here’s what happened.

The winner: Claude Fable 5

Claude produced the strongest founder-ready strategy memo.

Its biggest strength was that it made a clear strategic choice.

It did not recommend launching as a generic “AI assistant for small businesses”. Instead, it recommended a focused wedge into regulated micro-practices and privacy-sensitive professional services: accountants, solicitors, bookkeepers, financial advisers, HR consultants, consultants, and agencies handling confidential client data.

That was the sharpest insight in the whole comparison.

Its positioning was also the strongest:

>

That works because it does not try to out-feature Microsoft Copilot or Google Workspace. It reframes the competition around data custody, client confidentiality, and trust.

Claude’s best recommendation was: don’t compete on being cheaper than Copilot. Compete on privacy, control, and workflows that cloud-first incumbents cannot credibly own.

It also had the strongest risk analysis: Microsoft bundling, local model quality gaps, hardware variability, support burden, regulatory shifts, and category confusion with free local tools.

Overall, Claude felt the most client-ready.

GPT-5.5 was the best operator

GPT-5.5 came very close.

It was less punchy than Claude on positioning, but stronger on execution.

It produced the most practical 90-day launch plan: choose two verticals, run workflow audits, recruit pilot firms, configure 3 to 5 daily automations per customer, measure admin hours saved, build case studies, then convert pilots into paid customers.

It was also more cautious around compliance claims. That matters. A privacy-first AI product should avoid saying “GDPR-compliant by design” too casually. Better language is: “designed to reduce unnecessary data transfer and support UK GDPR obligations, subject to configuration.”

GPT-5.5 was very useful for turning the strategy into an operating plan.

If Claude gave the boardroom memo, GPT-5.5 gave the launch checklist.

Local Qwen was better than expected

The local qwen3.6:27b model produced a coherent, complete, and genuinely useful first draft.

It covered all required sections. It had a competitor table, pricing hypothesis, go-to-market phases, risk table, and launch plan. For a local model, it performed well.

But it had weaknesses.

It made more unsupported claims. It was less disciplined with citations. It overclaimed in places, for example saying local-first meant “zero data-privacy risk”, which is not accurate. Local-first reduces risk, but it does not eliminate it.

It also picked freelancers and micro-agencies as the primary beachhead. That is easier to market to, but less strategically defensible than privacy-sensitive professional services.

Still, the result was good enough for internal ideation, early drafting, and private strategy work.

That is important.

Local models do not need to beat frontier cloud models at everything to be useful. They need to be good enough for the right part of the workflow.

My ranking

1. Claude Fable 5 Best for strategy, positioning, founder-ready narrative, and final synthesis.
2. GPT-5.5 Best for launch planning, pilot design, pricing experiments, and operational detail.
3. qwen3.6:27b local Best for private first drafts, brainstorming, internal notes, and cheap iteration.

The bigger takeaway

The best workflow was not “pick one model”.

The best workflow was hybrid:

Use the local model first to brainstorm privately and cheaply.

Use GPT-5.5 to turn the ideas into a practical operating plan.

Use Claude to sharpen the positioning and produce the final client-ready narrative.

That feels like where AI work is heading.

Not one model for everything.

A portfolio of models, each used where it is strongest.

For privacy-first products especially, local models have a clear role. They are not always the best final writer. They are not always the strongest strategist. But they are useful for private thinking, early drafting, and working with sensitive material before anything goes to the cloud.

In this test, local Qwen was not the winner.

But it was absolutely good enough to be part of the team.

And that may be the more important result.

GitHub

I would like to share my latest open source local LLM inference tool implemented in C#. It supports models like Gemma4, Qwen3.6 with multi-modal (image, vision, audio), reasoning and function tool. It can run on Windows/MacOS/Linux and fully leverage GPU's capability. The API is completely compatible with OpenAI and Ollama interface.

Really appreciated if you can try it and give me some feedback. If you like it, it will be a big thank you if you can star it. Thank you very much!

Everyone benchmarks tool-calling with one number: "Model X gets 71% of function calls right." That number can't tell you why the other 29% failed — and the "why" is what decides what you do next.

So I built Toolhound, an MLX-native diagnostic that runs entirely on your Mac and attributes every tool-calling failure to one of four causes:

- `framework_template_bug` — the chat template mangled the tool tokens

- `framework_parser_gap` — the model emitted a rescuable call, the framework parser missed it

- `model_format_failure` — the model can't emit a parseable call

- `model_decision_failure` — valid format, wrong tool/args

What surprised me (Qwen2.5-0.5B / 1.5B, Llama-3.2-3B, 4-bit, on an M2 Pro):

- Qwen2.5-0.5B mostly fails on an upstream chat-template bug — Qwen2.5's template renders its tool-call example with doubled braces `{{"name": ...}}`, and the small model copies it literally. That's not the model's fault. args-correct 29%.

- Qwen2.5-1.5B parses fine (96%) but fails on judgment — wrong tool/args. args-correct 71%.

- Llama-3.2-3B formats perfectly, but wrong arg types + false abstentions. args-correct 61%.

Same benchmark, opposite root causes. A plain accuracy score hides that — and the smallest model's failures aren't even fixable by a better model.

Other things it does:

- 95% bootstrap CIs on every metric (temp=0, so no seed hand-waving — the CI comes from resampling the case set)

- Reports attribution under both a strict and a lenient parser, so you can see the verdict doesn't flip

- Quantifies bf16-vs-q4 damage without confounding it with template differences (asserts identical template first)

- v2 benchmarks existing zero-training fixes (PA-Tool is wired in). Honestly, on my demo run PA-Tool didn't beat baseline on any metric — it flags a result "credible" only when its CI is disjoint from baseline's, and it wasn't (it even hurt 1.5B's arg accuracy). I'd rather the tool tell me that than rubber-stamp it.

https://github.com/Code-byte404/toolhound

Feedback very welcome — especially: which models should I add next, and are the abstention "trap" cases too easy/hard? There are `good first issue's if anyone wants to add a model or help file the template bugs it finds upstream.

Builders-welcome post with the substance up front (disclosure: I'm the maintainer). OmniRoute is a free, MIT, self-hosted AI gateway — one OpenAI-compatible endpoint over 237 providers — built around two problems: runs dying on a provider 429, and tokens bleeding on tool/log output.

One endpoint, 237 providers — 90+ of them free. You point any tool or agent at a single OpenAI-compatible endpoint (localhost:20128/v1) and it can reach 237 LLM providers without you rewriting anything. 90+ have free tiers and 11 are free forever (no card), which aggregates to ~1.6B documented free tokens/month — and that's honest, pool-deduped math (we count each shared pool once instead of inflating it; the methodology is public in the repo). There's a one-command setup-* for 13+ coding tools (Claude Code, Codex, Cursor, Cline, Roo, Kilo, Gemini CLI…), so switching your existing setup over takes seconds.

Fallback combos — so it never stops mid-task. A "combo" is a ladder of models the router walks automatically: your subscription first, then API keys, then cheap models, then free ones. When a provider returns a 500 or you hit a rate limit, it slides to the next target in milliseconds, mid-request, and your tool never even sees the error. There are 17 routing strategies (priority, weighted, round-robin, cost-optimized, auto/coding:fast…) plus three resilience layers — a per-provider circuit breaker, a per-key cooldown, and a per-model lockout — so one dead key can't take down a whole provider.

Fusion — an ensemble mode for the hard steps. Beyond simple routing, there's a fusion strategy that fans a single prompt out to a panel of different models in parallel and then has a judge model synthesize one best answer (mixture-of-agents, built in). It's cost-aware, so easy turns stay on one fast model and it only fuses when the step is worth it.

A 10-engine compression pipeline — the part most routers don't have. Every request flows through a transparent compression pass you can toggle/stack per combo. Instead of one trick, it stacks the best of the open-source ecosystem: RTK filters command/tool output (git diffs, test logs, builds) at 60–90%, Microsoft's LLMLingua-2 does ML semantic pruning, Caveman handles prose, session-dedup strips repeats across turns. Critically, code, URLs and JSON are preserved byte-perfect, and a default-on inflation guard throws the compressed version away and sends the original if compressing would actually grow the prompt — it never makes things worse. On tool-heavy sessions that's ~89% average input-token reduction (an 8k-token git diff becomes a few hundred). Full credit to every upstream project (RTK, Caveman, LLMLingua-2, Troglodita) is in the README.

Agent-native — the agent can drive the router itself. There's a built-in MCP server (95 tools across 30 audited scopes, over stdio / SSE / streamable-HTTP), plus A2A (v0.3, JSON-RPC 2.0) support. That means an agent can query providers, switch combos, read its own remaining quota and manage memory through the gateway — not just consume tokens through it.

It's 100% local (zero telemetry, AES-256-GCM at rest), MIT-licensed, has a prompt-injection guard on every LLM route, opt-in memory, and runs on npm, Docker, desktop or your phone via Termux.

For context on whether it's worth your time: it's grown to ~9.8K GitHub stars, 1,490+ forks and 280+ contributors in ~4.5 months, with 21,000+ automated tests and 1,830+ issues closed — so it's a battle-tested project, not a brand-new experiment.

npm install -g omniroute

GitHub: https://github.com/diegosouzapw/OmniRoute · Site: https://omniroute.online

Would value a critique of the routing/compression architecture from this crowd.

Ask questions across your Markdown notes using a fully local Graph RAG engine. Built for Obsidian vaults, works with any folder of Markdown files. Extracts entity-relation triples from wikilinks & YAML frontmatter, retrieves answers via hybrid search (vector + BM25 + temporal). Multilingual. No cloud. Runs on Ollama.

https://github.com/benmaster82/Kwipu

Gemma 4 models are up to 90% faster with Ollama 0.31 on Apple Silicon. The speedup comes from multi-token prediction (MTP). This was achieved through contributions to the MLX kernel and the improvement isn't limited to Gemma 4 models. Gemma 4 is the first model to receive this improvement.

Note: You need to download new mlx tags to get this improvement.

Scope

Ollama's official benchmark measures Gemma 4 12B on the Aider polyglot benchmark using an M5 Max.

I wanted to run a simpler benchmark on an older device. I tested gemma4:e4b on a single JSON generation prompt. This means a different model size and a different workload. This isn't a validation or refutation of their numbers, just a separate data point on smaller hardware with a narrower task.

What I Tested

I compared gemma4:e4b and gemma4:e4b-mlx on M1 Pro MacBook Pro 16GB with a simple JSON array generation prompt. I picked this prompt because MTP especially performs better when output predictable (closing brackets, repeated identifiers, boilerplate).

Generation Speed (Higher is Better)

Prompt: "Generate a JSON array of 30 fake user objects with id, name, email, and signup_date fields."

Model	tokens/second
gemma4:e4b	32.60
gemma4:e4b-mlx	50.95

It performs 56% better with mlx on a 5 years old machine.

Conclusion

This is a meaningful improvement but because it comes from multi-token prediction, it mostly benefits predictable generations. That is, it won't give the same performance improvement for all prompts.

The app I used: Reins: Chat for Ollama

Read More: The Blog Post