We're building agents that can read millions of documents, but still forget a video they watched yesterday.
One thing has felt odd to me while working with AI agents.
We've gotten pretty good at giving them memory for text.
They can search documentation, index repositories, retrieve past conversations, and even build long-term memory over time.
Videos, though, are still treated as temporary input.
The agent watches a recording, answers a few questions, and when the session ends, that understanding is usually gone. Next session, the same video gets processed all over again.
That feels like an architectural gap rather than a model limitation.
A video isn't fundamentally different from any other source of information. Once you've extracted transcripts, OCR, visual observations, and timestamps, why throw that work away?
I ended up building an open-source project around this idea.
Instead of asking the agent to repeatedly "watch" the same video, it builds a persistent local index the first time. Future questions become retrieval instead of video analysis.
It changed how I think about video in agent workflows.
I'm curious whether others see this as a real missing piece, or if you've already solved it another way.
