u/Harag

A month ago I posted the very rough beginnings of a paper here. That version got pulled apart and rebuilt, and it is now a proper preprint: https://doi.org/10.5281/zenodo.21139628. The paper is deliberately language-neutral, though, and the system behind it is Common Lisp top to bottom: an orchestrator that runs execution graphs (the graphs themselves are plain Lisp data), agent worker loops, a process supervisor, a document store, a dashboard, and a set of MCP servers that give coding agents structured access to Lisp systems. So this follow-up is about the Lisp side, since this is the group that would ask.

The learnings that mattered most carry straight over from the paper. The first was composable domains. A domain is a bundle of instructions, skills, and tool access handed to an agent for a class of task: lisp knows the language, cl-naive knows the stack's conventions, a role adds the stance on top, and a resolver merges the layers per session. Built as one-offs they were dead weight. Redesigned to compose (stack cleanly, assume nothing about each other) they started turning up useful in places I had not planned for, and the same domains are now carrying a separate CL application build unchanged.

The second was the ratchet. An agent once delivered a load test asserting a record count was greater than or equal to zero: green forever, catches nothing, looks completely normal in review. So the loop runs backwards from instinct now. Acceptance criteria are written before the code exists, the coding agent writes no tests at all, a fresh session verifies the code against those criteria, another session then derives regression tests from them, and a judge breaks the implementation on purpose to confirm each test can actually fail. Standards move one way only. And the part I am most pleased about: the system develops itself through exactly this cycle, agents editing the orchestrator's own source through its own gates. It is the third generation of the design and only recently became my daily driver, so I will not oversell maturity.

The third was MCP tool naming, which mattered far more than it should. Models route off a tool's name, and the name drags their training priors with it. My file writer took code where every built-in file tool takes content; agents thrashed on it until I renamed the parameter. The code analyzer (definitions, callers, call graphs, effect sets over ASDF systems) sat ignored while agents grepped, until its descriptions spoke the vocabulary models already know from IDE land (Find All References, go to definition) and stated the economics plainly: the query is instant and exact, the text search is the slow, lossy path. One more rule from that fight: test your tool surface with the weakest model that can still do the work, because a strong one absorbs interface defects and hides them from you.

The fourth is the CL-only one: structural editing. Agents editing Lisp as raw text eventually break brackets, and then try to repair by counting parens, and that loop does not converge; even strong models ping-pong on it. So agents here do not edit text. The editor MCP exposes code as forms (replace this function's body, add this form after that one, validate before anything touches disk), bracket-correct by construction, courtesy of the reader treating code as data. A whole class of broken files simply stopped existing.

Everything is open, part of the opinionated cl-naive-* family (I came to Lisp late in my career, I care about usability over purity, and these libraries are geared toward shipping business software, not rocket science): https://gitlab.com/naive-x/experimental/cl-naive-full-stack-agentic-system. The preprint with the full methodology is here: https://doi.org/10.5281/zenodo.21139628. One question for this group: is anyone else giving their agents a persistent REPL to verify against, rather than only files and cold subprocesses?

A month ago I posted the very rough beginnings of a paper. That rough version did not survive: it got pulled apart and rebuilt by the very process it describes, and what came out the other side is now a proper preprint with a DOI: https://doi.org/10.5281/zenodo.21139628. Short version: the core claim held. The artifacts (specs, plans, executable graphs) and the verification gates wrapped around them have proven out on real work. Agents produce the work, the gates catch the defects, and a milestone only closes when the evidence is real, not when the model announces it is done.

Honestly, though, the headline result was not the most valuable thing I got out of building it. What I actually want to pass on is three things I learned making it work.

The first was composable domains. A "domain" in my setup is a bundle of instructions, skills, and tool access you hand an agent for a class of task. I built the first few as one-offs. Once I redesigned them to compose (stack cleanly, assume nothing about each other) they started turning up useful in places I had not planned for. A domain written for one workflow dropped straight into two others unchanged, and the same pattern is now carrying an entirely separate application build. Designing for composition instead of single use is the thing I would do first next time.

The second was the ratchet, and it needs a concrete example. An agent once delivered a load test asserting the record count was greater than or equal to zero. Green forever, catches nothing, and it looks completely normal in review. So the loop now runs like this: acceptance criteria are written before the code exists, the coding agent never writes tests at all, a fresh session verifies the code against those criteria, only then does another session derive regression tests from them, and a final step breaks the code on purpose to confirm each test can actually fail. A test that survives that is frozen, and later work runs against it and cannot silently undo it. Standards move one way only. That killed a whole class of "looks done, isn't."

The third was dumber and more surprising: tool naming matters far more than it should. An agent routes off a tool's name, and the name drags the model's training priors with it. What fixed things was never cleverness: borrow names from tools the model already knows, mirror the built-in parameter vocabulary exactly (renaming one parameter from `code` to `content` ended a whole class of thrashing), and never let a familiar name lie about what the tool does. The kicker: a strong model absorbs a bad interface and hides the problem from you, so test your tool surface with the weakest model that can still do the work.

Everything above runs as an open reference implementation: the orchestrator, the verification cycle, the composable domains. To set expectations, this is not another 180-line agent loop. It is the third generation of a design that got ground out until it was useful rather than until it was postable, and it has only recently earned daily-driver status. It also passes the dogfood test, since the system's own development runs through its own gates, and the deepest bugs it ever caught were in itself. Fair warning before you click: it is Common Lisp. https://gitlab.com/naive-x/experimental/cl-naive-full-stack-agentic-system

Preprint is here if you want the formal version: https://doi.org/10.5281/zenodo.21139628. Happy to take questions. And one worth asking of any agent-written suite: when did a test last fail because it caught wrong code? I could not answer that for mine, and that is where all of this started.