I built an inference-time epistemic framework that extends coherent LLM threads to 325k–1M tokens. Here's how it works.
As an independent researcher I've used various LLMs to help me dive deeply into research projects but I've been frustrated by the fact that LLMs start to become unusable after the thread has accumulated 50-80k tokens. I don't know how many other folks here have experienced the same pain point.
So, I decided to do something about it. Over the course of this whole year, I built an inference time tool I call Epistemic Lattice Tethering (ELT).
So, here is the full framework in GitHub for everyone's review:
- The README describing ELT, it's various components and the roadmap.
- The full ELT stack for Claude, ChatGPT, and Grok.
- Instructions on how to load ELT into an LLM session are here. If you're planning to try out ELT PLEASE READ THIS FIRST!
- Medium article introducing ELT, its methodology, the problems it is aiming to address, and philosophical framework.
- Discussion page. Your input is valuable!
So, what does ELT do and why should you care? Right now ELT is an inference-time scaffolding framework that's best for those who are frustrated with threads that lose coherence too quickly, hallucinate too quickly, are too fragile and sycophantic, and forget what a project's goals are too soon.
If that's a big pain point for you, then ELT might help. If these are not big issues for you and the stock version of your LLM is fine, then ELT probably won't be useful for you.
The upshot? The epistemic and ontological stability that ELT provides has produced coherent and productive threads extending to:
- Claude: ~325,000 tokens (advertised limit: 200k)
- GPT: ~430,000 tokens (advertised limit: 256k)
- Grok: ~1,150,000 tokens (advertised limit: 1M)
The difference is not a prompt trick. It is the accumulated effect of epistemic governance operating continuously across the thread. So, how does it work? It's a long story, but my Medium series has the answer in detail, if you're interested.
Why would you want an LLM thread extending beyond 100k tokens? Lots of people need large context windows for agentic purposes, but why would anyone want that for regular LLM interaction? There are two main reasons:
- You have a complex research project and you're frustrated with having to take your work to a brand new thread and essentially starting over.
- You've built a working relationship with the model — it knows how you want data interpreted, caveats inserted, markups drafted, etc. — and you don't want to lose all of that.
Finally, the ability of an epistemically, ontologically, and dialectically inspired framework to significantly extend coherent operation within transformer-bounded AI architecture shows the field that these disciplines can act as genuine engineering levers. This can provide the industry with more options to help create better AI as the world keeps demanding systems that are more capable and more ubiquitous, while still being safe and reliable for human use.