Why I aughta

EDIT 2 (14 Aug): Numbers below have changed. Following the judge-model point raised in the comments, I confirmed the 0.831 run was judged by gpt-4.1-mini, not the gpt-4o-mini the paper uses. Re-judged with the paper's judge, the figure is 0.8175. I also found my per-category labels were wrong — LoCoMo's categories are multi-hop / temporal / open-domain / single-hop, and I'd labelled them with LongMemEval's vocabulary, so what I called "multi-session" is temporal reasoning. Separately, 8 answers lost to an OpenAI outage mid-run had never made it into the saved output file; they're restored and the data now reproduces its own scores. Everything below reflects the corrected numbers, and the run data is now published so you can check it.

EDIT (13 Aug 22:41 GMT): A commenter correctly identified an error in the headline comparison. The 0.801 figure I used for Memora comes from Table 3 of the paper — a component build-up ablation row ("primary abstraction, with update"), not the published system result. Memora's actual scores from Table 1 are 0.849 (semantic retriever) and 0.863 (policy retriever). My 0.831 sits below both of those, so the headline as written overstates the comparison.

--

The numbers were run using Memora's (Microsoft Research, arXiv:2602.03315) open-source benchmark harness. The eval and scoring code is unmodified — I added a Recordari adapter for the memory backend. Same LoCoMo dataset, same category-exclusion convention (adversarial excluded, 1,540 scored questions), and the same models the paper specifies: gpt-4.1-mini for memory curation and answering (§5.1), gpt-4o-mini as the LLM judge (Appendix B).

Judge-matched, Recordari scores 0.8175 end to end against Memora's published 0.849 (semantic retriever) and 0.863 (policy retriever). It is below both.

Per category is the more interesting read. Against Memora S: multi-hop 0.801 vs 0.784, temporal 0.863 vs 0.851, open-domain 0.594 vs 0.594, single-hop 0.831 vs 0.900. Weighted by question count, the entire 0.032 gap is single-hop detail recall — extraction paraphrases specifics away ("salads, sandwiches and homemade desserts" becomes "dinner") where raw verbatim storage keeps them. Parity or better on everything else.

The internal before/after: raw batch storage 0.8065 → extraction 0.8175 overall, and on temporal questions 0.757 → 0.863. That gain comes from resolving relative dates to absolute at write time. Worth noting the aggregate difference (+0.011) is far smaller than the category one, and the judge model alone is worth 0.010–0.017, so I'd treat anything under ~0.004 as noise.

BLEU/F1 move the other way in Phase 2 (0.370/0.440 vs 0.464/0.547). Token-overlap metrics reward verbatim storage, so paraphrasing into clean facts costs surface overlap by design — flagging it here rather than leaving it in the README.

All results public (MIT): https://github.com/corbym/locomo-recordari — including the raw run data, both judges' per-question verdicts, and the score files, so you can re-score without paying for a run.

The harness runs against the prod API - exactly the MCP search and recall api that a real agent would use. Not a mocked backend, the real API, with a configuration of top_k=30, and a 1 hop edge expansion.

Recordari works with Claude, Claude Code, ChatGPT and pretty much any agent that can connect to MCP and reason.

If you want to run an agent against it to run the LoCoMo benchmark yourself, you can anon login at:

https://admin.recordar.io/start

Grab your personal key to use in the harness.

If you don't want to run the harness, just have a play with the sandbox memories, one click from the dashboard sets it up. Just remember to add the Full Skill from the Connect page, and then connect your agent.

--

What is recordari?

Recordari is a multi tenanted memory graph accessible by MCP. The graph can be used by teams, memories stay in the graph when disagreements happen, and resolve using type edges rather than being removed.

References and Further reading:

• Recordari documentation: https://docs.recordar.io/
• Recordari commercial site: https://recordar.io/
• Recordari Admin (register/sign in page): https://admin.recordar.io/
• Recordari Anon login (team tier): https://admin.recordar.io/start
• Microsoft blog: https://www.microsoft.com/en-us/research/blog/memora-a-harmonic-memory-representation-balancing-abstraction-and-specificity/
• Memora github: https://github.com/microsoft/Memora
• Memora paper: https://arxiv.org/abs/2602.03315

Things to ask the agent when running the demo sandbox:

• Why was VTIR created?
• Describe what was next in the project and why?
• Why must Pascal fixtures be written before the Rust port?
• What is TurboSound and why does it need special handling?
• What is the current state of the project?
• Why does WASM file I/O work differently from native?
• What are the standing rules for the AY chip port?

Song to download and try on VTIR

https://corbym.github.io/vtir/  (live web demo)

• https://zxart.ee/tune/325740  (download pt3 original file)

Memory doesn't work by place alone. Memories are chained. One day you smell freshly baked bread in a store, and you remember home. And home reminds you to call your mom. This is a narrative, a series of related things.

It's why a memory palace works for people. You imagine the places and the places remind you of where you put the items that remind you of the event. But the images are just placeholders for the story, a narrative you are creating to thread your memories together in useful ways. Agents don't work with images, they don't imagine drawers or rooms, they work with tokens and text.

• memoryweb remembers related information in nodes with narrative edges. The bug you fixed was because a user couldn't save a file. The broken file save was caused by a corrupt file system, which is why it needed fixing now. Without the "because", the node doesn't mean anything, and will be raised as drift candidates. This means memoryweb is a decision log, not an event log - events just grow, decisions are created with intent.

• memoryweb can "forget", too. Nodes can become stale, or contradictory, and forgetting is an action the agent can choose to take. They don't ever get purged unless you explicitly run a command, so every memory is always there, just not always surfaced.

• memoryweb can cause the agent to dream. Dreaming is just a hook that runs to show the agent what nodes are candidates for archive, and which nodes have been added recently. It helps orient the agent and keep the narrative fresh.

Please check it out, I don't think any other memory system works quite like this. I've used it with some success already and it's very useful to keep all related context fresh between sessions.

Let me know what you think!

https://github.com/corbym/memoryweb