Multi-agent handoffs and ghost context, what worked for me
say you've got multiple agents working on the same project, each doing a different job, lets say one's researching, one's writing code, one's handling qa, whatever your setup looks like. classic pattern is agent a finishes its part and passes a summary to agent b so it knows what's been done.
problem is that summary is lossy. agent a strips out raw detail to save tokens, agent b gets the compressed version and ends up guessing/hallucinating to fill whatever's missing. chain a few of these handoffs together and by the end, the last agent's working off something that's drifted pretty far from what actually happened...
what i tried that didn't really work
- shared markdown file both agents read/write to. sounds simple but burns a ton of tokens keeping it updated, and in practice the agents don't reliably "internalize" what's written there, especially mid task. they skim it more than actually track it.
- summarized handoffs between agents. the lossy compression problem above. works ok for short chains, falls apart fast once you've got 3+ agents touching the same work.
- vector cache as shared memory. better than nothing but still a compressed/approximate version of the actual state, so drift is still possible, just slower.
- mainstream tools. tried a few but they didn't really fit my workflow, ended up spending more time setting them up than actually getting work done.
so i decided to building a note/task app for the past few months and multi-agent coordination turned out to be the hardest part, way harder than i expected. built it specifically around this problem.
what i did was stop passing messages between agents entirely. instead gave them a shared structured state they both read/write to directly, so agent b never needs agent a to "tell" it anything, it just reads the real current state straight from source.
concretely this is a note-first pm app with an mcp server. workflow looks like this:
- you write a plan as a plain note, normal messy human writing, checkboxes for tasks
- agents connect via mcp and can read the note directly, not a summary of it, the actual text
- when an agent picks up a task, it claims it in the shared state using connected notes/references, so each agent gets its own task progress that all the other agents can still view and read if needed
- as it works, it updates status and writes results/logs back into the same note, not a separate handoff file
- next agent (or you) reading that note sees the real current state, not someone's compressed account of it
bonus part is it doubles as regular task management too, my tasks, my team's tasks, and the multi-agent workflow all sit in the same system, so each agent is basically treated like a team member. everyone (human or agent) can see task status, and it reads the team calendar to assign work with everyone's busy days as context.
so there's no handoff step at all, just one source of truth every agent hits directly. anything append-only/immutable (like a git commit log) beats a compressed summary that can drift from what actually happened, that's roughly the model i went with.
So what you guys are doing for this, especially once you're past 2 agents. centralized state, event sourcing, something else entirely?