We don’t trust LLMs to read an email properly. Why are we putting them in charge of entire workflows?
I keep seeing variations of the same complaints about LLMs:
“It didn’t read the whole email thread.” “It stopped halfway through.” “It skipped some of the work.” “It confidently told me something that wasn’t true.”
Fair complaints.
But then we do something I find slightly bizarre.
We ask the same systems to analyse a 40-page contract, modify a production codebase, research a market, operate a browser, handle company data, make decisions and run workflows unattended — then ask the LLM whether it successfully completed the job.
We apparently don’t trust LLMs with the small stuff, while increasingly trusting them with the big stuff.
I’m not convinced the answer is simply “wait for the next model”.
Maybe we have the architecture wrong.
A lot of current systems effectively ask the LLM to understand the task, remember the state, decide what happens next, choose and use tools, recover from errors — and finally determine whether its own work was correct.
That’s a remarkable amount of responsibility to give the least reliable component of the system.
So I’m increasingly interested in the inverse architecture:
Put state, memory, permissions, evidence, verification and workflow control outside the LLM.
Then use the LLM for what it’s actually good at: interpretation, reasoning, synthesis, creation and dealing with ambiguity.
In other words:
Maybe the LLM shouldn’t run the system. Maybe the system should run the LLM.
I’m much more interested in what people are actually doing about this than another discussion about which model currently tops which benchmark.
So, for people building real systems:
What do you actually do when the LLM lies, skips work, stops early, loses state or incorrectly claims success?
What have you moved outside the model?
State machines? Independent verification? Deterministic tests? Evals? Event logs? Evidence/provenance? Permission boundaries? Multiple models? External memory? Something else?
And what infrastructure do you wish existed but currently doesn’t?
One final provocation: if your primary method for determining whether an LLM completed its task correctly is asking the same LLM whether it completed its task correctly, I’m not sure you’re doing LLM engineering.
A better prompt or another edit to CLAUDE.md definitely isn’t the answer.
There is one basic engineering practice in particular that I think separates LLM engineering from LLM theatre.
What do you think it is?
And, more importantly, what are you actually using?
Co-written with my sparring partner, ChatGPT. Given the subject, disclosure seems appropriate. I won’t start crediting my MacBook and Wi-Fi.