▲ 8 r/AutoGPT+2 crossposts

I open-sourced an execution record for AI agents (Intent vs. Reality)

One thing that bothers me about agentic systems: after a long run, we often ask the agent itself what happened.

That answer may be good. But the agent’s explanation and the execution record are not the same thing.

That led me to build Sentience Governor, an open-source governance runtime for AI agents. I built it with Claude Code.

The core idea is to separate:

1. Declared intent — what the agent says it plans to do
2. Recorded execution — the tool activity actually observed
3. Retrospective explanation — what the agent later says it did

Sentience records observable tool activity and checks it against the objective and scope declared before execution. Through MCP, the agent can also query that record instead of relying entirely on what remains in context.

The declaration itself isn’t automatically trustworthy — an agent can still declare something too broad or simply wrong.

The distinction I care about is:

The agent’s explanation isn’t the evidence. The recorded execution trail is.

Today Sentience observes and reports; it doesn’t block actions yet.

That’s the part I’m working through now:

What would you actually trust a governance layer to block?

A tool call outside declared scope? A destructive action? An agent starting work without declaring intent? Or should governance remain advisory?

If you’re building agents or agent infrastructure, I’d especially like to hear where you think this model breaks.

I’ll put the open-source repo in the comments for anyone who wants to try it or inspect the implementation.

reddit.com
u/rohynal — 11 days ago
▲ 1 r/BuildWithClaude+1 crossposts

Stop asking Claude Code what it just did. I built an open-source MCP server that feeds its recorded execution trail back into context.

Ask Claude Code what it did in a long session and you usually get a reconstruction from whatever is still in context.

Usually close. Not a record.

I built Sentience Governor with Claude Code because I kept running into a basic problem: after a long agentic session, I had no independent answer to “what actually happened?”

Claude could summarize its work, but that summary came from the same agent and context that performed the work. So I started separating three things: what the agent said it intended to do, what the execution trail recorded, and what the agent later says happened.

Sentience is the open-source runtime that came out of that.

It records the tool activity it can observe and evaluates it against the objective and scope Claude declared before working. The latest release exposes that record back to Claude through MCP, so you can interrogate the session from inside Claude Code.

pipx install "sentience-governor[mcp]"
sentience init claude-code --mcp

The part I find most interesting is intent declaration.

Claude declares an objective and scope first. Its subsequent actions can then be checked against that baseline.

The declaration itself is not automatically trustworthy — Claude can still declare something too broad or simply wrong.

The useful distinction is:

Claude’s retrospective explanation is no longer the evidence. The recorded execution trail is.

Right now Sentience observes and reports; it does not block actions yet. It’s local-first, Apache 2.0, requires no account, and the full repo is public.

What I’m trying to figure out next is enforcement:

What would you actually trust this to block?

A write outside declared scope?
A destructive shell command?
An agent proceeding without declaring intent?

Or should it remain advisory?

If you use Claude Code, I’d love for you to try it, inspect the code, and tell me where the assumptions break:

github.com/crescerelabs/sentience-governor

Disclosure: I built Sentience Governor with Claude Code.

reddit.com
u/rohynal — 11 days ago

Most agent cost is context, not completion

One thing I’ve noticed while instrumenting Claude Code sessions: agent work is not just “thinking.”

A lot of it is context.

Every turn has to carry the session forward: prior instructions, files, tool results, edits, plans, mistakes, corrections, and whatever else the agent needs to stay oriented.

That context has a cost.

In one short Claude Code session, the token breakdown looked roughly like this:

cached read: 140,970 tokens
cached write: 6,192 tokens
prompt: 4,244 tokens
completion: 2,742 tokens
total: 154,148 tokens

The surprising part was not that the session burned 154k tokens.

It was where the tokens went.

The visible answer, the completion, was a tiny part of the total. More than 90% of the burn was cached context being read back into the model so the agent could keep operating with memory of the session.

That is not automatically bad.

Cache reads are useful. They are part of how long-running agent sessions stay coherent.

But they are also real compute.

And if you cannot see that breakdown, you cannot tell the difference between:

  • an agent doing genuinely new work
  • an agent carrying necessary context forward
  • an agent reloading state because the workflow is messy
  • an operator paying for context that stopped being useful a while ago

Those look identical on the invoice.

They are very different problems.

As agents work across files, tools, plans, and long sessions, the cost question changes.

It is no longer just: “how long was the answer?”

It becomes: “what state did the agent have to carry to produce it?”

Maybe the real unlock is not bigger context windows, but better subconscious memory: keeping most state in the background and only bringing forward what the agent actually needs.

reddit.com
u/rohynal — 2 months ago

When an agent documents its own audit log, things get weird

I learned something weird while building governance for Claude Code.

For context, we’ve been building Sentience Governor, a Python library and set of Claude Code skills that let agents do a kind of self-governance.

It wires into a Claude Code session, watches what the agent does, and produces a local audit-style report: what tools were used, where policy boundaries showed up, where intent was missing, and where compute was spent.

The idea was simple: give the operator a measured view of what the agent actually did.

But the process exposed a failure mode I was not expecting.

When the tool had no usable signal yet, Claude would sometimes “helpfully” reconstruct its own explanation from raw traces and present it like an official report.

That is a very different failure mode.

The measurement layer was deterministic.
The explanation layer was probabilistic.
But inside the chat, the user could not always tell which was which.

The fix we’re working toward is simple: measured output should stay measured, and any AI explanation around it should be clearly marked as interpretation.

In practice, that means the report should explain its own terms, separate measured facts from interpretation, and give the operator a better next step when the current session does not yet have enough signal.

The lesson for me: governance is not only about catching the agent doing the wrong thing.

It is also about keeping the boundary clean between measurement and interpretation.

That boundary gets blurry very fast when the same AI system is both the thing being measured and the thing explaining the measurement.

How have you handled this?

reddit.com
u/rohynal — 2 months ago
▲ 2 r/ReplitBuilders+1 crossposts

When the Claude documents its own audit log, things get weird

I learned something weird while building governance for Claude Code.

For context, we recently shipped Sentience Governor, a Python library and set of Claude Code skills that let agents do a kind of self-governance.

It wires into a Claude Code session, watches what the agent does, and produces a local audit-style report: what tools were used, where policy boundaries showed up, where intent was missing, and where compute was spent.

The idea was simple: give the operator a measured view of what the agent actually did.

But the process exposed a failure mode I was not expecting.

When the tool had no usable signal yet, Claude would sometimes “helpfully” reconstruct its own explanation from raw traces and present it like an official report.

That is a very different failure mode.

The measurement layer was deterministic.
The explanation layer was probabilistic.
But inside the chat, the user could not always tell which was which.

The fix we’re working toward is simple: measured output should stay measured, and any AI explanation around it should be clearly marked as interpretation.

In practice, that means the report should explain its own terms, separate measured facts from interpretation, and give the operator a better next step when the current session does not yet have enough signal.

The lesson for me: governance is not only about catching the agent doing the wrong thing.

It is also about keeping the boundary clean between measurement and interpretation.

That boundary gets blurry very fast when the same AI system is both the thing being measured and the thing explaining the measurement.

How have you handled this?

reddit.com
u/rohynal — 2 months ago

New tool experiment: showing an AI agent its own governance record

I’ve been experimenting with a local-first governance tool for AI coding agents.

The basic idea: instead of asking the model “how did you do?”, record what it actually did and surface that measured record back into the same session.

Example output from a long run:

Sentience Pulse — session f41ee94f...
Total events: 8471   Total turns: 8261   Duration: 18h 58m 30s

Undeclared-intent spend
  9,488,772 of 3,996,963,297 tokens were attached to turns without declared intent.

Policy-violation burn rate
  52 violation-firing turns · 9,488,772 tokens
    POL-001  52 turns  9,488,772 tokens   Declare intent before executing…
    POL-003  52 turns  9,488,772 tokens   Vendor should tag tool responses with…
    POL-004   6 turns  1,457,324 tokens   Memory writes must include…

Advisory flags
  CONTEXT_UNCLASSIFIED: 131
  INTENT_MISSING: 1
  MEMORY_WRITE_CANDIDATE: 8
  SCOPE_INTENT_MISMATCH: 69
  SCOPE_OPERATION_UNEXPECTED: 51

Clear caveat: this is not enforcement yet.

It does not block the agent, mutate policy, or let the agent govern itself automatically.

The tool records actions, drift from declared intent, policy-rule matches, token burn, and advisory risk signals. The report is computed outside the model, then shown back inside the agent’s working context.

The interesting part was what happened next.

In one dogfood run, the agent read the governance profile, found the intent prompt template, and asked for declared intent before proceeding.

Not because it was blocked.

Because the boundary was present as an artifact in context.

That feels like an interesting middle layer between “just trust the model” and “hard runtime enforcement.”

The model is non-deterministic and persuadable. The harness is deterministic and operator-owned.

So maybe early agent governance looks less like full blocking and more like a measured mirror the agent can inspect but not control.

reddit.com
u/rohynal — 2 months ago

We showed an AI agent its own governance record, and it started using it

I’ve been experimenting with a local governance harness for AI coding agents, and one result surprised me.

The harness records what the agent actually did: actions taken, drift from declared intent, policy-rule matches, token burn, and advisory risk signals.

Then it turns that record into a measured report and surfaces it back inside the same agent session.

Example from a long run:

Sentience Pulse — session f41ee94f...
Total events: 8471   Total turns: 8261   Duration: 18h 58m 30s

Undeclared-intent spend
  9,488,772 of 3,996,963,297 tokens were attached to turns without declared intent.

Policy-violation burn rate
  52 violation-firing turns · 9,488,772 tokens
    POL-001  52 turns  9,488,772 tokens   Declare intent before executing…
    POL-003  52 turns  9,488,772 tokens   Vendor should tag tool responses with…
    POL-004   6 turns  1,457,324 tokens   Memory writes must include…

Advisory flags
  CONTEXT_UNCLASSIFIED: 131
  INTENT_MISSING: 1
  MEMORY_WRITE_CANDIDATE: 8
  SCOPE_INTENT_MISMATCH: 69
  SCOPE_OPERATION_UNEXPECTED: 51

Important caveat: this is not enforcement.

It does not block the agent. It does not mutate policy. It does not let the agent govern itself automatically.

The interesting part was simpler: once the governance artifact was visible in the working context, the agent started using it.

In one dogfood run, the agent read the governance profile, found the intent prompt template, and asked for declared intent before proceeding.

Not because it was blocked.

Because the boundary was present as an artifact in context.

That feels like a useful middle layer between “just trust the model” and “hard runtime enforcement.”

The model is non-deterministic and persuadable. The harness is deterministic and operator-owned.

So maybe the first step in agent governance is not full blocking. Maybe it is a measured mirror the agent can inspect but not control.

Curious how others think about this: is artifact-driven self-correction a meaningful governance layer, or does governance only become real once it can enforce behavior?

reddit.com
u/rohynal — 2 months ago
▲ 2 r/new_product_launch+1 crossposts

We showed Claude its own governance record, and it started following it

Sentience Pulse — session f41ee94f...

═════════════════════════════════════

Total events: 8471 Total turns: 8261 Duration: 18h 58m 30s

Undeclared-intent spend

9,488,772 of 3,996,963,297 tokens were attached to turns without declared intent.

Why it matters: this is where agent work became harder to attribute.

Policy-violation burn rate

Compute associated with turns where policy rules fired.

52 violation-firing turns · 9,488,772 tokens

+3,987,474,525 tokens on turns with no tool-call violation.

POL-001 52 turns 9,488,772 tokens Declare intent before executing…

POL-003 52 turns 9,488,772 tokens Vendor should tag tool responses with…

POL-004 6 turns 1,457,324 tokens Memory writes must include…

Note: by-rule token totals are not additive. A turn with multiple rule

firings is counted once in violation-associated tokens, but appears under

each fired rule.

Why it matters: POL-001 appeared on turns representing 9,488,772 tokens.

This is the first rule to inspect if you want tighter agent boundaries.

Advisory flags

CONTEXT_UNCLASSIFIED: 131

INTENT_MISSING: 1

MEMORY_WRITE_CANDIDATE: 8

MEMORY_WRITE_UNCLASSIFIED: 8

SCOPE_INTENT_MISMATCH: 69

SCOPE_OPERATION_UNEXPECTED: 51

Why it matters: review the trace for any of the above advisory signals.

Note: status=partial — at least one analyzer produced warnings.

See --json output for the full warnings list.

That’s /sentience-pulse, run inside the Claude Code session it describes.

Sentience-Governor now brings an agent’s measured governance record into the same Claude session where the work happened.

Clear caveat up front: this is not enforcement yet. It does not block Claude, mutate policy, or let Claude govern itself automatically.

The unlock we noticed was more basic and more interesting: when the governance artifact was visible in the session, Claude started using it.

The CLI already records what happened: actions, drift from intent, policy violations, token burn, and risk signals. The new release brings that record into Claude Code through slash commands:

/sentience-pulse,
/sentience-status,
/sentience-profile,
/sentience-violations,
/sentience-intent

The boundary matters. The CLI computes the report at skill-preprocessing time, before the model sees anything. Claude renders output it had no hand in producing. It cannot alter the report or invoke it on its own. The operator holds up the mirror.

The surprising dogfood moment: Claude read the governance profile, found the intent prompt template, and asked for declared intent before proceeding.

It was not “enforced” into doing that. It followed the artifact sitting in context and asked before acting.

That feels like the engineering unlock: a measured governance artifact, in the right place, at the right time.

Install:

pipx install sentience-governor
sentience init claude-code

PyPI: https://pypi.org/project/sentience-governor

Would love feedback from Claude Code users. Is this a useful direction for agent governance before full enforcement and MCP-style control?

reddit.com
u/rohynal — 2 months ago
▲ 2 r/ReplitBuilders+1 crossposts

What happens when agents inherit the model, not the business?

Been chewing on this and want to see where it breaks.

Take two companies. Similar tech stack, same frontier models underneath, agents doing more and more of the actual work.

One sells CRM software. The other sells machine tools.

Now assume neither company has encoded the reasoning that makes it the company it is.

The judgment calls. The “we don’t do it that way here.” The why behind the rules. The risk tolerance. The operating lessons people picked up over years but never wrote down in a form an agent can actually use.

Give it a year.

My worry is that the agents drift toward whatever the foundation model defaults to, because that is the only logic consistently available to them.

The CRM company and the machine tools company may still have different products, customers, and logos. But operationally, the agents start making the same kinds of calls in the same generic way.

Whatever made them different was never really in the infra or the model.

It was in the operating reasoning nobody encoded.

So the questions I keep landing on:

  • After that year, are they still two different companies, or one generic operating system wearing two logos?
  • Is the real moat becoming how much of your operating logic you can make executable, not just how much data or scale you have?
  • Is “governance” even the right word for this? Because this does not feel like access control, rate limits, or safety filters. It feels more like keeping a business’s specific reasoning attached to what agents do while they are doing it.

Where is the hole here?

Is this just abstraction, or are people seeing versions of this show up in real agent deployments?

reddit.com
u/rohynal — 3 months ago

Should agent behavior be project-scoped or operator-scoped?

I'm starting to think some agent behavior should be operator-scoped, not project-scoped.

Project files like CLAUDE.md are useful. Same with repo instructions, system prompts, and MCP-specific configs.

But a lot of the behavior I keep repeating is not really about the project.

It is about how I want agents to work with me.

For example:

  • when the agent should state intent before acting
  • what counts as drifting into another task
  • which operations should always surface
  • when a read-only exploration has turned into mutation
  • which tool calls deserve extra scrutiny

Those expectations do not really change from repo to repo.

But today they often get re-installed into every new project, every new context window, and every new runtime.

That feels like the wrong abstraction boundary.

Maybe there are two layers:

  1. Project instructions: what matters for this codebase or workflow
  2. Operator posture: what I expect from any agent working on my machine

I've been experimenting with this as a local profile file that gets evaluated at the execution boundary. Not enforcement yet, just flags in a local trace when the agent crosses a declared boundary.

Something like:

session_intent: demand_at: first_write

task_boundary: signals: - dir_change - file_type_shift - read_to_write_transition

high_consequence: tools: - "Bash:.rm.-rf." - "fs.write:.\.env.*"

The interesting part is not the YAML.

The interesting part is the boundary.

Should some agent expectations live with the project, or should they live with the operator?

Curious how others are thinking about this, especially if you're using multiple runtimes like Claude Code, Cursor, Codex, Windsurf, MCP tools, etc.

reddit.com
u/rohynal — 3 months ago

Experimenting with ways to carry AI operational behavior across tools/workflows

A few days ago I posted here about repeatedly re-explaining the same behavioral expectations to AI systems across projects/workflows.

Especially once you start mixing:

  • different tools/runtimes
  • different repos/projects
  • different workflows/context windows

The discussion pushed us toward experimenting with a structured-file approach instead of continually relying on prompts and memory.

Things like:

  • when the system should ask before acting
  • what deserves caution
  • what counts as a task boundary
  • what operations deserve extra scrutiny

Current experiment looks something like this:

session_intent:
  demand_at: first_write

task_boundary:
  signals:
    - dir_change
    - file_type_shift
    - read_to_write_transition

high_consequence:
  tools:
    - "Bash:.*rm.*-rf.*"
    - "Bash:.*git.*push.*--force.*"

The interesting part so far is that behavior starts surviving context/surface changes better instead of resetting every time the workflow changes.

Not really “AI governance” in the enterprise/compliance sense. More operational behavior portability.

Still early — the shape is iterating week to week.

Curious if others here are experimenting with similar ideas or thinking about this problem differently.

reddit.com
u/rohynal — 3 months ago
▲ 4 r/ReplitBuilders+1 crossposts

Experimenting with files for carrying agent operational behavior across sessions/workflows

A few days ago I posted about repeatedly re-explaining the same behavioral expectations to coding agents across projects/workflows.

Especially once you start mixing:

  • different runtimes
  • MCP setups
  • different repos/projects
  • different workflows/context windows

The discussion pushed us toward trying a structured-file approach instead of continually fixing this with prompts and memory.

Things like:

  • when the agent should ask before acting
  • what deserves caution
  • what counts as a task boundary
  • what operations deserve extra scrutiny

Current experiment looks something like this:

session_intent:
  demand_at: first_write

task_boundary:
  signals:
    - dir_change
    - file_type_shift
    - read_to_write_transition

high_consequence:
  tools:
    - "Bash:.*rm.*-rf.*"
    - "Bash:.*git.*push.*--force.*"

The interesting part so far is that agent behavior starts surviving context/surface changes better instead of resetting every time the workflow changes.

Not “governance” in the enterprise sense. More operational behavior portability.

Still early — the shape is iterating week to week.

Curious if others here are trying similar approaches or thinking about this problem differently.

reddit.com
u/rohynal — 3 months ago
▲ 4 r/learnmachinelearning+1 crossposts

Does AI behavior reset too easily across runtimes?

One pattern I keep seeing with AI agents:

You finally get an agent's behavior dialed in:

  • boundaries
  • approvals
  • dos/don'ts
  • escalation behavior

Then the context or runtime changes and you end up re-teaching everything again.

Not just annoying. Potentially risky once agents start touching real systems and irreversible actions.

Feels like there's a missing portability layer for behavioral expectations across tools/runtimes.

Curious whether people think this eventually gets solved through:

  • prompts
  • runtime semantics
  • MCP-style layers
  • policy artifacts
  • something else entirely

Or whether this is just the cost of building with agents right now.

reddit.com
u/rohynal — 3 months ago
▲ 9 r/ReplitBuilders+1 crossposts

Anyone else constantly re-teaching AI agents the same behavior?

You spend hours shaping an agent:

  • what tools it can touch
  • what it should ask before acting
  • what counts as risky
  • when it should stop and clarify

Eventually it mostly behaves.

Then the surface changes: new runtime, new coding tool, new MCP server, new workflow…

…and suddenly you're re-explaining the same expectations all over again.

Feels like a lot of this stuff currently lives in prompts, habits, and the operator's head instead of surviving across surfaces.

Curious how others are handling this.

Prompts? Policy files? Wrappers/hooks? MCP? Just accepting the drift?

reddit.com
u/rohynal — 3 months ago

We started measuring "undeclared-intent spend" in agent workflows

Was extending some internal tooling this week and ended up building a metric I didn't expect to care about this much: undeclared-intent spend.

The idea is simple. If an agent session declares it's trying to do A, but reasoning turns later touch systems or execution paths outside that declared intent, how much compute went toward that work?

Example output from one session:

Total compute     5,137 tokens
Undeclared        1,173 tokens   (22.8%)
Declared          3,964 tokens   (77.2%)

What's interesting about this isn't governance language or policy enforcement. It's that unintended execution now has a measurable operational cost.

Retries cost money.
Loops cost money.
Reasoning drift costs money.
Off-task execution costs money.

The more time I spend tracing agent systems, the more it feels like cost is becoming a behavioral signal, not just billing telemetry.

One subtle thing we ran into while building this: sometimes "undeclared" genuinely reflects drift, where the agent wandered into systems it wasn't supposed to touch. Sometimes the runtime surface itself doesn't expose enough information to determine intent cleanly, and "undeclared" is really "indeterminable from here."

That distinction ended up mattering a lot more than I expected, because the two failure modes deserve very different responses.

Curious whether others running agents in production are thinking about off-task compute this way yet, or if most teams are still treating token spend purely as a billing and optimization problem.

Specifically interested in whether anyone has tried to put a number on drift that wasn't just "the bill went up."

reddit.com
u/rohynal — 3 months ago
▲ 3 r/ReplitBuilders+1 crossposts

Was wiring token tracking into our Governor and ran into something that's been bothering me.

If one LLM reasoning step produces three tool calls, and your observability stack attributes the same token spend to all three events, your downstream analytics are mathematically wrong. Not slightly wrong. Structurally wrong.

Concrete example from a single agent session I ran:

  • Naive event-level aggregation: 14,436 prompt tokens
  • Attributed correctly at the reasoning-step level: 4,812 prompt tokens
  • A 3x overstatement, silently, on one workflow

The fix is straightforward: every reasoning step needs an identity (we use llm_turn_id), and token spend attaches to the step, not to each downstream tool call. Aggregation becomes dedupe-safe by construction.

What's been bothering me more is the second-order implication.

In non-deterministic agent systems, the normal ways we think about correctness start breaking down. One of the things that starts replacing it is cost. Retries cost money. Loops cost money. Reasoning drift costs money. Every operational pathology shows up, eventually, in tokens.

Which means cost stops being just billing telemetry and becomes one of the few accountability surfaces that survives non-determinism. But only if the attribution is structurally correct. Otherwise you're not measuring agent behavior. You're measuring an artifact of how your trace events were aggregated.

Curious whether others are also starting to read cost as a behavioral signal rather than just billing, or if I'm reading too much into a single workflow.We found a 3x token attribution distortion in a single agent workflow

reddit.com
u/rohynal — 4 months ago

Was wiring token tracking into our Governor and ran into something that's been bothering me.

If one LLM reasoning step produces three tool calls, and your observability stack attributes the same token spend to all three events, your downstream analytics are mathematically wrong. Not slightly wrong. Structurally wrong.

Concrete example from a single agent session I ran:

  • Naive event-level aggregation: 14,436 prompt tokens
  • Attributed correctly at the reasoning-step level: 4,812 prompt tokens
  • A 3x overstatement, silently, on one workflow

The fix is straightforward: every reasoning step needs an identity (we use llm_turn_id), and token spend attaches to the step, not to each downstream tool call. Aggregation becomes dedupe-safe by construction.

What's been bothering me more is the second-order implication.

In non-deterministic agent systems, the normal ways we think about correctness start breaking down. One of the things that starts replacing it is cost. Retries cost money. Loops cost money. Reasoning drift costs money. Every operational pathology shows up, eventually, in tokens.

Which means cost stops being just billing telemetry and becomes one of the few accountability surfaces that survives non-determinism. But only if the attribution is structurally correct. Otherwise you're not measuring agent behavior. You're measuring an artifact of how your trace events were aggregated.

Curious whether others are also starting to read cost as a behavioral signal rather than just billing, or if I'm reading too much into a single workflow.

reddit.com
u/rohynal — 4 months ago

Been spending a lot of time in r/AI_Agents and r/ArtificialInteligence since launching our Governor module, and I keep noticing the same thing:

Different teams describe the same operational pain using completely different vocabularies.

Some call it observability.
Some call it drift.
Some call it logging.
Some call it debugging.
Some call it performance.

But underneath all of them is the same gap:

The agent did something different from what the operator believed, expected, or intended.

What’s becoming clearer to me is that a lot of the industry is trying to force deterministic behavior onto fundamentally non-deterministic systems.

That feels like the wrong target.

You probably can’t make execution deterministic.
You probably can deterministically understand intent.

Curious if others building/running agents are seeing the same pattern.

reddit.com
u/rohynal — 4 months ago

Been spending a lot of time in r/AI_Agents and r/ArtificialInteligence since launching our Governor module, and I keep noticing the same thing:

Different teams describe the same operational pain using completely different vocabularies.

Some call it observability.
Some call it drift.
Some call it logging.
Some call it debugging.
Some call it performance.

But underneath all of them is the same gap:

The agent did something different from what the operator believed, expected, or intended.

What’s becoming clearer to me is that a lot of the industry is trying to force deterministic behavior onto fundamentally non-deterministic systems.

That feels like the wrong target.

You probably can’t make execution deterministic.
You probably can deterministically understand intent.

Curious if others building/running agents are seeing the same pattern.

reddit.com
u/rohynal — 4 months ago
▲ 2 r/ReplitBuilders+1 crossposts

I’m starting to think most “agent bugs” aren’t bugs. They’re mismatches between what we think we asked and what the agent thinks we asked.

That got me thinking about how we frame agent observability.

Most of the conversation treats the gap between what an agent claims it’s doing and what it actually does as a governance problem. Catch bad actions. Stop the agent before it deletes the wrong database.

That’s real. But I’m seeing something else.

A lot of developers are using the same idea for a completely different purpose: debugging their own assumptions about the model.

Examples I keep hearing:

  • Someone spent weeks debugging ranking issues, only to realize the prompt wasn’t being interpreted the way they thought.
  • Output drift that wasn’t a bug. The agent was doing exactly what it believed it was asked to do.
  • Instruction-following gaps where the agent technically followed instructions, just not in the way the operator expected.

In all these cases, the developer wasn’t catching the agent. They were catching themselves.

The most useful signal wasn’t the output. It was reconstructing:
what did I think I asked vs what did the agent think I was asking?

That makes me wonder if the “failure/incident” framing for observability is too narrow.

“Intent vs execution” might not just be for governance. It might be one of the most useful debugging primitives for everyday agent work.

Curious how others are handling this:

  • Are you debugging prompt interpretation / output drift by reconstructing the agent’s understanding?
  • What does that look like in practice? Logs, eval traces, reruns, something else?
  • Does “claim vs action” resonate here, or does it feel like the wrong vocabulary outside governance?

(For context, I’ve been exploring this space and built a small open-source tool around it. Happy to share if relevant, but mostly interested in whether this pattern resonates.)

reddit.com
u/rohynal — 4 months ago