r/WTFisAI

I built an open-source, self-hosted AI gateway: 237 providers (90+ free), auto-fallback combos, and a 10-engine token-compression pipeline (MIT)
▲ 336 r/WTFisAI+69 crossposts

I built an open-source, self-hosted AI gateway: 237 providers (90+ free), auto-fallback combos, and a 10-engine token-compression pipeline (MIT)

Builders-welcome post with the substance up front (disclosure: I'm the maintainer). OmniRoute is a free, MIT, self-hosted AI gateway — one OpenAI-compatible endpoint over 237 providers — built around two problems: runs dying on a provider 429, and tokens bleeding on tool/log output.

One endpoint, 237 providers — 90+ of them free. You point any tool or agent at a single OpenAI-compatible endpoint (localhost:20128/v1) and it can reach 237 LLM providers without you rewriting anything. 90+ have free tiers and 11 are free forever (no card), which aggregates to ~1.6B documented free tokens/month — and that's honest, pool-deduped math (we count each shared pool once instead of inflating it; the methodology is public in the repo). There's a one-command setup-* for 13+ coding tools (Claude Code, Codex, Cursor, Cline, Roo, Kilo, Gemini CLI…), so switching your existing setup over takes seconds.

Fallback combos — so it never stops mid-task. A "combo" is a ladder of models the router walks automatically: your subscription first, then API keys, then cheap models, then free ones. When a provider returns a 500 or you hit a rate limit, it slides to the next target in milliseconds, mid-request, and your tool never even sees the error. There are 17 routing strategies (priority, weighted, round-robin, cost-optimized, auto/coding:fast…) plus three resilience layers — a per-provider circuit breaker, a per-key cooldown, and a per-model lockout — so one dead key can't take down a whole provider.

Fusion — an ensemble mode for the hard steps. Beyond simple routing, there's a fusion strategy that fans a single prompt out to a panel of different models in parallel and then has a judge model synthesize one best answer (mixture-of-agents, built in). It's cost-aware, so easy turns stay on one fast model and it only fuses when the step is worth it.

A 10-engine compression pipeline — the part most routers don't have. Every request flows through a transparent compression pass you can toggle/stack per combo. Instead of one trick, it stacks the best of the open-source ecosystem: RTK filters command/tool output (git diffs, test logs, builds) at 60–90%, Microsoft's LLMLingua-2 does ML semantic pruning, Caveman handles prose, session-dedup strips repeats across turns. Critically, code, URLs and JSON are preserved byte-perfect, and a default-on inflation guard throws the compressed version away and sends the original if compressing would actually grow the prompt — it never makes things worse. On tool-heavy sessions that's ~89% average input-token reduction (an 8k-token git diff becomes a few hundred). Full credit to every upstream project (RTK, Caveman, LLMLingua-2, Troglodita) is in the README.

Agent-native — the agent can drive the router itself. There's a built-in MCP server (95 tools across 30 audited scopes, over stdio / SSE / streamable-HTTP), plus A2A (v0.3, JSON-RPC 2.0) support. That means an agent can query providers, switch combos, read its own remaining quota and manage memory through the gateway — not just consume tokens through it.

It's 100% local (zero telemetry, AES-256-GCM at rest), MIT-licensed, has a prompt-injection guard on every LLM route, opt-in memory, and runs on npm, Docker, desktop or your phone via Termux.

For context on whether it's worth your time: it's grown to ~9.8K GitHub stars, 1,490+ forks and 280+ contributors in ~4.5 months, with 21,000+ automated tests and 1,830+ issues closed — so it's a battle-tested project, not a brand-new experiment.

npm install -g omniroute

GitHub: https://github.com/diegosouzapw/OmniRoute · Site: https://omniroute.online

Would value a critique of the routing/compression architecture from this crowd.

u/ZombieGold5145 — 3 days ago
▲ 160 r/WTFisAI+27 crossposts

How to build an AGY WIKI OKF on the Antigravity CLI

AGY Builders,

We are all trying to build useful and scalable workflows for our AGY CLI and ecosystem, but the speed at which we need to learn, build, and deploy new things is incredibly overwhelming. If you are feeling that pressure, you are in the right place here at r/GoogleAntigravityCLI.

Over the past few weeks, I have been testing an "AGY WIKI OKF" setup that I put together myself (after inviting some members of this community to collaborate; mod is not proud). I know some folks might hesitate to trust a tutorial from a random Redditor, but I wanted to share this with the community anyway because it actually works.

I was able to build this because I am all-in on Google and the Antigravity Ecosystem. I’m a truly AGY—I am not some ultra-smart, 10x developer, but I know how to work hard, I dig for the right information, and I iterate.

AGY WIKI OKF | The Idea

To build a frictionless, token-efficient knowledge WIKI engine that transforms static documentation or notes (information) into an active, intelligent collaborator—orchestrated entirely by Antigravity CLI.

The core philosophy is simple: treat knowledge management as a clean pipeline and tokens as a premium, finite resource.

By anchoring this architecture to Google’s Antigravity CLI, the AGY WIKI OKF bypasses heavy middleware and complex UI layers, delivering a hyper-focused AI partner built entirely for execution speed, context hygiene, and minimal footprint.

Why adopting AGY WIKI OKF matters:

  • Stay organized (AGY OCD): Structured Markdown and YAML keep the chaos in check.
  • Save tokens: Doing more with less context window bloat.
  • Scale shareable knowledge: Making it easy to pass context and logic between different LLMs.
  • Humans and Agents working together: One standardized, readable format that works perfectly for both of us.
  • BYOD (Bring Your Own Data): Own your context. Port it to the newest model, platform, or OS instantly.

The Tools

The WIKI

In the agent-first era, a WIKI is no longer just a static graveyard for human notes; it is the operational hard drive for your agents. By maintaining a highly structured WIKI, you ensure that every piece of context is stored in a clean, machine-readable format. This means that whether you are testing a new modular skill or spinning up a specialized agent, your AGY CLI knows exactly where to find the precise context it needs to generate autonomous action, moving you far beyond simple, reactive conversational text.

Reference: Gist on Knowledge Representation

Google Open Knowledge Format (OKF)

Google’s Open Knowledge Format (OKF) feels like the exact missing piece we've needed for orchestrating multiple AI agents effectively. It provides a vendor-neutral, interoperable standard for storing and sharing organizational knowledge.

Why this is huge for orchestration:

  1. The "Lingua Franca" for Agents: Any agent can read it out of the box without platform-specific integrations.
  2. Seamless Context Passing: Specialized agents can access, update, and pass the exact same foundational context back and forth.
  3. Human-in-the-Loop Oversight: Because OKF is just Markdown and YAML, it’s inherently readable and auditable.
  4. Scalable Knowledge: It acts as a shared, living library that grows alongside your agents.

AGY WIKI OKF Integration

Structuring an AGY Wiki using OKF revolutionizes how complex knowledge is shared. By standardizing documentation with concise Markdown and YAML frontmatter, OKF provides a unified taxonomy for cataloging AGY CLI slash commands or skills It is highly token-efficient, stripping away bloated formatting and maximizing context window limits.

The Prompt for Building an AGY WIKI OKF

AGY CLI WIKI OKF PROMT EXAMPLE

/grillme I want to initialize a brand-new, empty Obsidian vault from scratch that adheres strictly to the Open Knowledge Format (OKF) standard, with the specific intent of potentially open-sourcing or sharing this architecture later. I want a purely blank, skeletal framework with no pre-populated data. Please grill me to define the optimal architectural blueprint for this vault. I need you to interrogate me on: Do not generate the directory structure or files until you are satisfied that you have captured all my requirements for a production-ready, shareable knowledge base. 
Core Directory Hierarchy: How should we structure the root (e.g., /concepts, /resources, /indices, /log) to be intuitive for external users? Template Strategy: What base boilerplate templates do we need to ensure every new file is automatically OKF-compliant and structured for consistent metadata? Workflow Logic: Since this is a fresh start, what processes should we bake in for capturing information vs. refining knowledge that could be easily documented for others? CLI Integration: What specific file locations or configurations do we need to ensure this vault plays nicely with the Antigravity CLI from day one? Open-Source & Contributor Documentation: What files should we create to make this a "deployable" standard? Please include requirements for: A README.md with installation and usage instructions. A CONTRIBUTING.md that defines how to add new concepts or schemas. A "System Architecture" document that explains the logic behind the folder structure and metadata fields, ensuring anyone who clones this vault understands how to extend it.

The Final File Structure

AGY WIKI OKF
    ├── .agyrc
    ├── ARCHITECTURE.md
    ├── CONTRIBUTING.md
    ├── README.md
    ├── .agy
    │   └── .keep
    ├── .obsidian
    │   ├── app.json
    │   ├── appearance.json
    │   ├── core-plugins.json
    │   └── workspace.json
    ├── 00-Inbox
    │   └── .keep
    ├── 10-Projects
    │   └── .keep
    ├── 20-Areas
    │   └── .keep
    ├── 30-Resources
    │   ├── .keep
    │   └── Google Antigravity Documentation.md
    ├── 40-Archive
    │   └── .keep
    ├── 99-Meta
    │   └── Templates
    │       ├── Base_Template.md
    │       ├── Project_Template.md
    │       └── Resource_Template.md
    └── Clippings

TL;DR

  • AGY WIKI OKF: Organizes your information (context) , AGY CLI commands, skills  behaviors, and A2A workflows into a token-efficient, shareable format that reduces inference costs for any LLM.
  • Open Knowledge Format (OKF): Provides a standardized, vendor-neutral way to share context (Markdown + YAML), preventing platform lock-in and eliminating data fragmentation.

AGY Builders, I genuinely want your input on this. Please comment, grill me, roast me, ask questions, or give me your raw feedback on this AGY WIKI OKF setup. We are building the foundation to organize and share our data in the BYOD era. Let's build the future together.

u/AgentPadrino — 3 days ago

Fable 5 is back today, three weeks after the government forced Anthropic to pull it worldwide

Anthropic's most powerful model, Fable 5, is back online today. The US government forced it offline about 3 weeks ago, and now the ban's been lifted.

Quick recap, some researchers at Amazon found a way to trick Fable 5 into writing code that could break into software. The government told Anthropic to lock it down, and since they couldn't check where each user was sitting fast enough, they pulled it for everyone on the planet.

It's back today, but they added a stricter safety filter that's a bit too cautious. If you use Fable 5 for regular coding or debugging, it'll sometimes decide your request looks risky and send it to a different Anthropic model, Opus 4.8, instead. So part of your everyday coding won't even run on the model you actually picked.

There's also a second model here, Mythos 5, the same underlying AI with fewer safety limits. That one isn't coming back for everyone, only a short list of approved US companies get it after a government review.

When Anthropic tested other models, OpenAI's GPT-5.5 and even their own Opus 4.8 could find the same security holes. So whatever got Fable 5 pulled wasn't really unique to it.

If you build on top of these tools like I do, watching one vanish overnight and return with new limits you didn't ask for is a strange feeling. You don't own it, and it can change under you with no warning.

Anyone else had a tool you rely on get pulled or changed out from under you?

u/DigiHold — 5 days ago

We open-sourced a Cloudflare Worker that tells you the moment an AI sends you a real visitor

We kept wanting to know the second a real person landed from ChatGPT, Perplexity, or Gemini, and GA4 just doesn't show it reliably: it needs the referer or utm to survive and the tag to fire, so most land in Direct.

So we built it and open-sourced it. A small Cloudflare Worker that reads each request at the edge and pings your phone (ntfy, Telegram, Discord, Slack...) when someone arrives from an AI answer. It can alert on AI crawlers too, off by default since they're noisy.

Free, MIT, no tracking script or database.

Repo: https://github.com/surfacedby/ai-traffic-alerts-for-cloudflare

u/SurfacedBy — 5 days ago
▲ 61 r/WTFisAI

Anthropic says Alibaba created 25,000 fake accounts to copy how Claude thinks

Anthropic just accused Alibaba's AI lab of the biggest copycat operation they've ever caught.

Between April and early June, someone allegedly created about 25,000 fake accounts on Claude and ran 28.8 million conversations with it. All those answers then got fed into Alibaba's own AI, called Qwen, teaching it to respond like Claude without doing the original research.

Anthropic had already called out similar stuff from other Chinese AI labs earlier this year, but nothing close to this size. Alibaba hasn't said a word about it publicly.

But this affects more than just Anthropic. Every one of those fake accounts used Claude the exact same way a real paying customer would, just thousands of times over in a coordinated system.

I personally love Claude, I use it every day for my work, but I kind of like this situation for one specific reason.
If Chinese labs become better at creating AI models, even if they stole data from popular US companies, it means that Claude, ChatGPT, Gemini, etc, will need to step up, maybe with better pricing or limitations, because how to keep users if you can have way cheaper with better limitations that do exactly the same things?
US labs will try to put upfront privacy "Chinese labs use and sell your data", probably true, but US companies too, so that's not an argument that will make users pay hundred each month to stay with them.

u/DigiHold — 7 days ago
▲ 121 r/WTFisAI

Google spent $2.7 billion to get one AI researcher back, and he just left again for OpenAI

Google lost two of its most important AI researchers in the same week, and both went straight to competitors.

Noam Shazeer co-wrote "Attention Is All You Need", the 2017 research paper that basically invented the technology behind ChatGPT, Claude, and every AI chatbot you've used. He left Google back in 2021 because he thought they moved too slow, started his own company called Character.AI, and Google paid $2.7 billion to bring him back in 2024. He stayed less than two years and just left for OpenAI.

Then there's John Jumper, who won the 2024 Nobel Prize in Chemistry for creating AlphaFold, an AI that predicts how proteins fold inside your body (which helps scientists develop new drugs way faster). He'd been at Google DeepMind for nine years and just left for Anthropic, the company behind Claude.

A third researcher named David Silver, one of DeepMind's earliest employees, also left recently to start his own company.

Google's stock dropped over 5% after the news came out.

Anyone else feel like Google went from leading the AI race to watching its best people walk out the door?

u/DigiHold — 10 days ago
▲ 33 r/WTFisAI

First Claude Fable 5, now the White House wants to approve who gets the next ChatGPT first

OpenAI's next ChatGPT model, GPT-5.6, was supposed to launch like any other update. Instead the White House asked OpenAI to hold it back and only let a small group of approved partners use it first, with the government signing off on who gets access one customer at a time.

The reason they gave is security, this new model is apparently as capable as Claude Mythos model, so the government wants to test it for safety before regular people can get their hands on it.

So a few weeks back, Claude Fable 5, had two of its newest AIs held up the exact same way. Now it's OpenAI's turn, so we're slowly landing in a world where the government decides when a new AI is allowed to reach the public, not the company that built it.

Testing powerful tools before they go live sounds reasonable, but needing permission before normal people can use the new model feels very weird to me.

u/DigiHold — 10 days ago
▲ 2 r/WTFisAI+4 crossposts

You shipped an AI project. Congrats. Now turn it into what investors, conferences, and Big Tech actually pay you thousands for

Imagine this.

You built something with AI. A model. A pipeline. A prototype.

You tweet it. 200 likes. Maybe a few DMs.

And then... nothing.

Now imagine a different timeline :

Same project. But this time, instead of just tweeting, you run a single command. Five questions later, you have a publication-ready research paper. Two-column PDF. Professional typesetting. Every claim verified. Every citation real.

You submit it to NeurIPS. It gets accepted.

Suddenly you're in a room with researchers from DeepMind, OpenAI, Meta. A VC who saw your paper asks for a meeting. "We read your work. We want to invest."

Six months later, you've turned that paper into a conference talk, a magazine article in MIT Tech Review, and a 300-page book with exercises and a glossary.

A year later, a big tech company acquires your team. Not because of your GitHub stars. Because of your publication record.

All of that started with a single document.

Most people don't know this world exists

The ceiling most builders see:

▎ Build → Tweet → Move on

The actual ceiling:

▎ Paper → Conference → Investors → Acquisition → Book

And the gap between those two? One properly written, properly verified document.

Here's why a paper matters more than you think:

Investors don't read pitch decks first. They Google you. If they find a published paper with verified claims, you get the meeting. A YC partner literally said: "500 AI startups a month. The ones with a published paper get a second meeting."

Big Tech doesn't acquire products. They acquire teams. How do they evaluate a team? Publications. It's the closest thing to a certification in this industry.

Conferences aren't just academic. NeurIPS, ICML, CVPR : these rooms are filled with people who will be your next co-founder, investor, or acquirer. A GitHub repo doesn't get you in. A paper does.

A paper is a flywheel. Paper → conference talk → magazine article → keynote → book deal. Each step makes the next one automatic.

The catch

AI-generated papers are garbage.

They invent statistics. They cite papers that don't exist. They make grand claims with nothing backing them up. After 100 pages, they start repeating themselves. Section titles end up alone at the bottom of pages.

You cannot send that to a conference. You cannot show it to an investor.

What I built

KORRO Research doesn't just help you write. It enforces correctness.

Here's what it actually catches:

Your paper says: "Our method achieves 78.4% accuracy, outperforming the state of the art."

→ The hallucination detector flags it. Where's the evidence? Where's the citation? Either prove it or remove it.

Your paper says: "As shown by Smith et al. (2023)..."

→ The citation verifier checks CrossRef and arXiv. Does that paper actually exist? If the DOI doesn't resolve, flagged. No fake references.

Your paper says in Chapter 2: "Our framework uses a three-stage pipeline." Then in Chapter 5: "The four-stage architecture..."

→ The consistency engine catches it. Three-stage or four-stage? Across 300 pages, every term is tracked. Nothing contradicts.

Your section title lands alone at the very bottom of page 4.

→ The typography engine catches it before PDF generation. No orphan headings. Ever.

Your paper doesn't say which GPU you used or what random seed.

→ The reproducibility engine won't let you export until the checklist is filled. Every modern conference requires this.

You need to switch from casual startup tone to NeurIPS anonymous academic style.

→ One command. --preset neurips. Every "we" becomes "the authors". Terminology, formatting, tone — all switched. No rewriting.

The result

python scripts/wizard.py

Five questions. Sixty seconds.

You get a PDF + DOCX + markdown skeleton.

You fill in your content. Run the engines. Fix what they flag.

What comes out is not just well-written. It's a document where every claim is classified, every stat is checked, every citation is real, every term is consistent, every page is typeset, and every venue requirement is met.

This isn't just for papers

Raising money? → Pitch deck. Financial model. Competitor matrix. YC, Sequoia templates.

Applying for a grant? → NSF, NIH, ERC, DARPA formats. Budget, timeline, broader impact.

Writing a book? → 10 chapters. Exercises. Glossary. Index. 300 pages, zero contradictions.

Publishing in Nature? → 150-word abstract. Data statement. Interdisciplinary tone.

Giving a talk? → Slides. Speaker notes. Timing. Q&A prep. PowerPoint, Keynote, Beamer.

One command. Nine formats. Zero hallucinations.

Try it

github.com/KorroAi/korroresearch

MIT license. Open source.

Clone it. Run the wizard. Write about what you built.

Even if you never submit the paper, you'll understand your own work better than before. The process forces you to support every claim and explain every decision.

And if you DO submit it?

That's when the doors open.

TL;DR : A research paper gets you into rooms that code alone can't. KORRO Research automates the entire pipeline: wizard → PDF + DOCX → 8 verification engines → venue-ready output. Every claim verified. Every citation real. Nine formats. MIT license. github.com/KorroAi/korroresearch

u/korro_ai — 9 days ago
▲ 20 r/WTFisAI

For the first time ever, less than half of AI users are on ChatGPT

Until recently, ChatGPT was the only AI tool most people had ever heard of. Your coworker used it, your dentist probably tried it, and nobody else was even close.

The latest industry numbers just came out and ChatGPT is down to 46% of the AI assistant market, the first time it's ever fallen below half. Google's Gemini is the biggest reason, because it's baked into Gmail, Search, Android, and basically everything Google already runs, so people aren't "switching" to it on purpose, it's just already there when they open their phone. And Claude quietly quadrupled its users without most people even noticing it existed.

But ChatGPT isn't actually shrinking, it still has over a billion people using it every month and that number keeps climbing. The whole market just grew so fast underneath it that holding majority share became impossible, kind of like how Chrome dominated browsers until Edge and Safari slowly chipped away at it by being pre-installed on every device.

Most people don't even know which AI they're talking to anymore. Google answers your search with AI before you click anything, and Microsoft sticks Copilot into every product it ships. Nobody is choosing an AI the way they chose an iPhone, the AI is just showing up wherever people already spend their time.

Feels like we're heading toward AI being more like email, where nobody cares which provider runs it as long as it works. Anyone else bouncing between different AIs without really thinking about which one you opened?

u/DigiHold — 13 days ago