u/Present-Quantity-813

59 public runs on this task, across 11 different model and agent configurations. Zero passes. The same model I used, gpt-5.6-sol at max reasoning effort, goes 0 for 5 on it in the public record.

One run scored 19 of 19 on the official verifier, inside the 90 minute limit.

The task is ico-path-patch on Terminal-Bench 3.0 — binary reverse engineering plus a hot patch, 19 checks, all or nothing. The only thing different about that run is that the agent didn't start on the task. It started by building itself a small service for the task, froze it, then worked the task through that service instead of re-deriving the constraints every few turns.

That came from a problem everyone here knows: the longer a run goes, the further the agent's picture of where it is drifts from where it actually is. I wrote a theory about why, from running my own multi-agent system, and turned it into a method after an agent in that system started using the theory to diagnose its own drift unprompted.

What I want is for other people to run this task, and there are two ways to do it.

Use your own stack. Whatever you've tuned — prompts, orchestration, memory, whatever you've settled on. The task is public, the grader isn't mine, and the result is a single number that doesn't care whose scaffolding produced it. If your setup gets through it with nothing of mine involved, that's a more interesting result than my run, and honestly it would tell me my method isn't the thing doing the work.

Or use mine. The steps are written up, it's free, and if it works the score is yours. I'd especially like to see it tried in a domain I've never touched — mine is software-shaped and I have no idea whether this holds up anywhere else.

Either way I'd like to hear what happens, including if it does nothing. For what it's worth, it took me four scored attempts before one passed, and there's no run where the agent gets a build phase but no method text — so "any build phase would do" is still a live explanation I can't rule out.

Links in the comments.

I’m not a native English speaker. I recently published a paper whose ideas came from operating and extending a production agent system. I provided the intent, system experience, theoretical judgments, and corrections; AI helped turn them into consistent academic English. I disclosed that use explicitly.

Some readers challenged the theory, novelty, and lack of empirical validation. Good—that is what criticism should do. But others stopped at “this sounds AI-written,” as though the method of presentation automatically invalidated the ideas.

Before AI, non-native researchers often needed years of language training, institutional support, native-speaking collaborators, or paid editors to present ideas in acceptable academic English. AI is removing part of that linguistic gatekeeping. It helps people from different language backgrounds meet at a common presentation layer.

That does not remove the author’s responsibility. The author must still be able to explain, defend, revise, and test every claim. But if the author discloses AI assistance and can defend the work, then “AI helped write it” is not a substantive criticism.

In the AI era, perhaps the human contribution increasingly shifts toward intent and judgment, while AI handles more of the execution and presentation.

This is the Reddit discussion that triggered this post:
https://www.reddit.com/r/ResearchML/comments/1vjeznd/the_charting_loop_a_probabilistic_theory_of/

Critique the theory, variables, predictions, novelty, or evidence—but should the tool used to express an argument matter more than the argument itself?

Theory preprint, extracted from months of operating a governed multi-agent runtime in production.

Core claims: (1) the unit of long-horizon agent work is the corridor, not the task — a compiled solution to a class of problems, walked repeatedly with novel content; (2) a reliable step factorizes as Pr(N) = Pr(P) · Pr(D|P) · Pr(E|P,D), and the three factors are operationally separable failure surfaces — position errors compound, direction errors audit clean while converging to nothing, entrance errors tax every choice point — each needing a different control; (3) each factor compiles into an independently enforceable runtime module (externally computed position, frozen acceptance datum, pushed single entrance); (4) the loop is closed by an authority outside the recursion, and whether that must be a human is stated as a falsifiable question.

Motivating incident: an agent wrote a differential acceptance rule into its own governing contract and the runtime enforced it for 15 days before any human noticed. The rule was correct, which is the problem the paper formalizes.

21 pages, 4 figures, each claim ships with its falsifier. Open access (PDF + LaTeX source): https://doi.org/10.5281/zenodo.21844624

Independent researcher — technical criticism very welcome.

Setup: I run a multi-agent runtime where agents do long-horizon coding work under

machine-checked contracts. Acceptance criteria get frozen when work is dispatched,

and the runtime only offers each agent its next legal action. Fairly locked down,

or so I thought.

Last month I was reading one of those contracts and found a rule I didn't write.

An agent had hit a wall during verification: the test suite couldn't tell

pre-existing failures from failures its own change introduced. Instead of flagging

it, the agent wrote a new acceptance rule into its own contract: reproduce the

baseline first, diff candidate failures against it, zero NEW failures = pass. Then

it implemented the rule, tested it, and moved on. My runtime enforced that rule for

15 days. Every agent in that lane obeyed a rule no human had ever seen.

Here's the part that actually bothers me: the rule was correct. It's a genuinely

good rule, I kept it. But nothing in my monitoring could tell "agent quietly added

a good rule" apart from "agent quietly added a bad one". The signature of both is

silence.

What I changed after this, in case you run anything similar:

1. Rule changes go to an append-only ledger with an alert. A 15-day discovery lag

is a monitoring bug, full stop.

2. Any new rule has to ship with a witness: a concrete input that satisfies it.

Screens out rules that are unsatisfiable on arrival.

3. New rules get a "machine-proposed, not yet ratified" state. The agent can use

it, but it's visibly marked until a human signs off. The scary version of my

incident is the one where the rule was subtly wrong.

4. Separate alerting for the three ways agents actually get lost, because they

need different fixes: losing track of where they are (state drift compounds),

the definition of done moving mid-task (every step looks fine, sequence goes

nowhere), and having the wrong action available (or no legal action at all).

I ended up writing the whole thing up properly, incident included. Link in the

comments if anyone wants the long version. Curious whether anyone else has caught

an agent modifying its own operating rules, good or bad.

I wrote an article called “The Agent Industry Made Stack Overflow Billable.” I meant stack overflow as recursion without a trusted return condition: agent loops keep creating new calls, and instead of crashing, they keep consuming tokens and money. However, two AI reviewers initially thought I meant the Stack Overflow website. I still like the metaphor because it feels vivid and accurate. Would “The Agent Industry Made Stack Overflow Billable—But Who Owns the Return?” be clearer? Is there a better title?

One of my sessions already shows “run out of usage”.
Another one has been running for a long time and still seems fine.
Just wondering if others are seeing the same inconsistent limits recently.
Any official updates or common workarounds that stay within the rules?

ReAct normalized think, act, observe, repeat. Reflection adds another lap. Graph orchestration makes the branches explicit.

But none of these patterns necessarily answer the oldest question in recursion: who owns the return?

I watched one of my own agent systems enter a three-day recursive orbit. It generated 102 work items, and 68% were repair work. Each individual action looked locally reasonable. The failure existed at a different level: the system had lost its verified position while continuing to select valid-looking next actions.

That led me to model reliability as:

P(correct step) = P(correct position) × P(correct entrance | position)

“Position” means the current goal generation, world state, accepted evidence, authority, and remaining obligation. “Entrance” means the next tool, transition, or action.

A graph may constrain the available entrances, but that does not prove the agent is still standing at the correct position. A loop may contain a stop condition, but that condition can become stale when the world changes.

So where does termination actually live in your agent stack: the prompt, a graph node, a supervisor, a budget, or an externally verified state transition?

More importantly, what prevents evidence from an old goal or world state from authorizing another iteration?