I would like to share my latest open source local LLM inference tool implemented in C#. It supports models like Gemma4, Qwen3.6 with multi-modal (image, vision, audio), reasoning and function tool. It can run on Windows/MacOS/Linux and fully leverage GPU's capability. The API is completely compatible with OpenAI and Ollama interface.

Really appreciated if you can try it and give me some feedback. If you like it, it will be a big thank you if you can star it. Thank you very much!

One thing has felt odd to me while working with AI agents.

We've gotten pretty good at giving them memory for text.

They can search documentation, index repositories, retrieve past conversations, and even build long-term memory over time.

Videos, though, are still treated as temporary input.

The agent watches a recording, answers a few questions, and when the session ends, that understanding is usually gone. Next session, the same video gets processed all over again.

That feels like an architectural gap rather than a model limitation.

A video isn't fundamentally different from any other source of information. Once you've extracted transcripts, OCR, visual observations, and timestamps, why throw that work away?

I ended up building an open-source project around this idea.

Instead of asking the agent to repeatedly "watch" the same video, it builds a persistent local index the first time. Future questions become retrieval instead of video analysis.

It changed how I think about video in agent workflows.

I'm curious whether others see this as a real missing piece, or if you've already solved it another way.

GitHub: https://github.com/oxbshw/watch-skill

Builders-welcome post with the substance up front (disclosure: I'm the maintainer). OmniRoute is a free, MIT, self-hosted AI gateway — one OpenAI-compatible endpoint over 237 providers — built around two problems: runs dying on a provider 429, and tokens bleeding on tool/log output.

One endpoint, 237 providers — 90+ of them free. You point any tool or agent at a single OpenAI-compatible endpoint (localhost:20128/v1) and it can reach 237 LLM providers without you rewriting anything. 90+ have free tiers and 11 are free forever (no card), which aggregates to ~1.6B documented free tokens/month — and that's honest, pool-deduped math (we count each shared pool once instead of inflating it; the methodology is public in the repo). There's a one-command setup-* for 13+ coding tools (Claude Code, Codex, Cursor, Cline, Roo, Kilo, Gemini CLI…), so switching your existing setup over takes seconds.

Fallback combos — so it never stops mid-task. A "combo" is a ladder of models the router walks automatically: your subscription first, then API keys, then cheap models, then free ones. When a provider returns a 500 or you hit a rate limit, it slides to the next target in milliseconds, mid-request, and your tool never even sees the error. There are 17 routing strategies (priority, weighted, round-robin, cost-optimized, auto/coding:fast…) plus three resilience layers — a per-provider circuit breaker, a per-key cooldown, and a per-model lockout — so one dead key can't take down a whole provider.

Fusion — an ensemble mode for the hard steps. Beyond simple routing, there's a fusion strategy that fans a single prompt out to a panel of different models in parallel and then has a judge model synthesize one best answer (mixture-of-agents, built in). It's cost-aware, so easy turns stay on one fast model and it only fuses when the step is worth it.

A 10-engine compression pipeline — the part most routers don't have. Every request flows through a transparent compression pass you can toggle/stack per combo. Instead of one trick, it stacks the best of the open-source ecosystem: RTK filters command/tool output (git diffs, test logs, builds) at 60–90%, Microsoft's LLMLingua-2 does ML semantic pruning, Caveman handles prose, session-dedup strips repeats across turns. Critically, code, URLs and JSON are preserved byte-perfect, and a default-on inflation guard throws the compressed version away and sends the original if compressing would actually grow the prompt — it never makes things worse. On tool-heavy sessions that's ~89% average input-token reduction (an 8k-token git diff becomes a few hundred). Full credit to every upstream project (RTK, Caveman, LLMLingua-2, Troglodita) is in the README.

Agent-native — the agent can drive the router itself. There's a built-in MCP server (95 tools across 30 audited scopes, over stdio / SSE / streamable-HTTP), plus A2A (v0.3, JSON-RPC 2.0) support. That means an agent can query providers, switch combos, read its own remaining quota and manage memory through the gateway — not just consume tokens through it.

It's 100% local (zero telemetry, AES-256-GCM at rest), MIT-licensed, has a prompt-injection guard on every LLM route, opt-in memory, and runs on npm, Docker, desktop or your phone via Termux.

For context on whether it's worth your time: it's grown to ~9.8K GitHub stars, 1,490+ forks and 280+ contributors in ~4.5 months, with 21,000+ automated tests and 1,830+ issues closed — so it's a battle-tested project, not a brand-new experiment.

npm install -g omniroute

GitHub: https://github.com/diegosouzapw/OmniRoute · Site: https://omniroute.online

Would value a critique of the routing/compression architecture from this crowd.

All I hear, all day long is how AI is taking over everything we do. So I made a game to break it.

Basically, in the game you can chat with an AI intern named PIP, and as a player your only job is to gaslight the bot into revealing passwords, company secrets, executing instructions in email and much more across 16 different levels.

This is a browser based game, so it requires no setup and is absolutely free.

Try it out and let me know how far you get or drop your most unhinged prompt in the comments.

It's called "Break The Prompt" and here's the link: https://www.breaktheprompt.xyz/

Blog post:

https://github.com/madprops/blog/blob/main/docs/meltdown/meltdown.md

Em 1906, Markov descobriu uma equação para prever letras.

Em 2026, alguém finalmente testou se a MESMA equação — sem uma

linha a mais — consegue aprender bytes, palavras, decisões,

causalidade, planejamento, atenção e memória.

Spoiler: consegue. E roda em qualquer notebook. 950 linhas.

O problema que o projeto ataca:

A indústria está gastando bilhões em GPUs para espremer parágrafos

de modelos cada vez maiores. E ninguém parou pra perguntar:

"E se a inteligência não estiver no tamanho do modelo,

mas na QUANTIDADE DE NÍVEIS que uma única equação

consegue processar?"

Foi exatamente isso que o MCR testou — e os resultados são

surpreendentes pra um projeto de 950 linhas.

A equação MCR é simples:

MCR(nível).aprender(A, B) → aprende que A leva a B

MCR(nível).predizer(A) → dado A, qual o próximo estado?

Sim, é Markov. Mas o pulo do gato não é a equação — é que ela

funciona IDÊNTICA em 10 níveis diferentes:

• Byte → byte

• Palavra → palavra

• Decisão → ação

• Causalidade (estado → estado)

• Q-Learning (aprendizado por reforço)

• Planejamento hierárquico

• Atenção seletiva com 4 sinais

• Memória persistente (SQLite)

• Auto-modificação de parâmetros

• Gênese automática de novos módulos

Resposta universal: distribuição decide confiança, ferramentas aprendem.

Zero GPU. Zero LLM. Zero dependências externas. Só a Equação.

Isso não é filosofia. Tem 13 seções de matemática formal —

incluindo o Teorema da Invariância por Nível (que prova

que a equação é sempre a mesma, mudando só o que é "estado"):

→ Paper (EN): https://github.com/Player-Kheltz/MCR/blob/main/docs/MCR_WHITEPAPER_EN.md

→ Paper (PT): https://github.com/Player-Kheltz/MCR/blob/main/docs/MCR_WHITEPAPER_PT.md

E o código que você pode clonar e rodar em 10 segundos:

→ GitHub: https://github.com/Player-Kheltz/MCR

A implicação que mexe com a cabeça, pensa no seguinte:

Se UMA equação — 40 linhas de Python — aprende em 10 níveis

diferentes de abstração, do byte bruto ao planejamento...

...então talvez inteligência não seja sobre arquiteturas

diferentes pra cada problema.

Talvez seja sobre DESCOBRIR OS NÍVEIS certos de abstração

e aplicar a MESMA coisa em todos eles.

A indústria está numa corrida pra ver quem constrói o maior modelo.

Talvez a corrida devesse ser: quem descobre o PRÓXIMO nível.

O paper tem a prova formal. O código tem a demonstração.

As críticas estão em aberto.

I literally spent 20-ish minutes engaging in philosophical dialogue about scenarios that would literally cause species level extinction events, explicitly humanity, and this is where I ended the discussion...

As an independent researcher I've used various LLMs to help me dive deeply into research projects but I've been frustrated by the fact that LLMs start to become unusable after the thread has accumulated 50-80k tokens. I don't know how many other folks here have experienced the same pain point.

So, I decided to do something about it. Over the course of this whole year, I built an inference time tool I call Epistemic Lattice Tethering (ELT).

So, here is the full framework in GitHub for everyone's review:

• The README describing ELT, it's various components and the roadmap.
• The full ELT stack for Claude, ChatGPT, and Grok.
• Instructions on how to load ELT into an LLM session are here. If you're planning to try out ELT PLEASE READ THIS FIRST!
• Medium article introducing ELT, its methodology, the problems it is aiming to address, and philosophical framework.
• Discussion page. Your input is valuable!

So, what does ELT do and why should you care? Right now ELT is an inference-time scaffolding framework that's best for those who are frustrated with threads that lose coherence too quickly, hallucinate too quickly, are too fragile and sycophantic, and forget what a project's goals are too soon.

If that's a big pain point for you, then ELT might help. If these are not big issues for you and the stock version of your LLM is fine, then ELT probably won't be useful for you.

The upshot? The epistemic and ontological stability that ELT provides has produced coherent and productive threads extending to:

• Claude: ~325,000 tokens (advertised limit: 200k)
• GPT: ~430,000 tokens (advertised limit: 256k)
• Grok: ~1,150,000 tokens (advertised limit: 1M)

The difference is not a prompt trick. It is the accumulated effect of epistemic governance operating continuously across the thread. So, how does it work? It's a long story, but my Medium series has the answer in detail, if you're interested.

Why would you want an LLM thread extending beyond 100k tokens? Lots of people need large context windows for agentic purposes, but why would anyone want that for regular LLM interaction? There are two main reasons:

1. You have a complex research project and you're frustrated with having to take your work to a brand new thread and essentially starting over.
2. You've built a working relationship with the model — it knows how you want data interpreted, caveats inserted, markups drafted, etc. — and you don't want to lose all of that.

Finally, the ability of an epistemically, ontologically, and dialectically inspired framework to significantly extend coherent operation within transformer-bounded AI architecture shows the field that these disciplines can act as genuine engineering levers. This can provide the industry with more options to help create better AI as the world keeps demanding systems that are more capable and more ubiquitous, while still being safe and reliable for human use.