u/Future_AGI

Midweek thread. Developers: the part of your LLM or agent setup you wish you could see more clearly.

- What is still a black box for you right now: retrieval, a tool call, token cost, why a run went sideways?
- What did you add a trace or a metric for recently, and did it actually earn its keep?
- What do you check first when something looks off, and what do you wish you could check instead?

Wins, dead ends, and half-formed "is it just me?" questions all welcome. New here?

Say what you are building and what you can and cannot see about it right now, and someone will have a take.

You give an agent a few tools and point it at a task. The answer comes back right, the output looks clean, and it feels ready to ship. Then you scroll through the trace just to be sure, and the middle of the run is a mess.

This pattern is common in tool-using agents. A research agent does search, fetch, summarize. The final summary is correct, but the fetch step pulled the wrong URL and the search fired twice on the same query. The model reached a right answer anyway, ignoring the junk it pulled and leaning on what it already had. Change the input slightly and that same broken path returns a wrong answer, with no obvious reason why.

The problem is that grading only the final output lets it through. The output is correct, so nothing gets flagged. Every mistake in the middle stays invisible, even though the trace has all the evidence.

What catches it is scoring each tool call against what it was supposed to do, not just grading the final answer. A right answer built on a wrong step should not count as a pass.

How are you catching mid-chain tool-call failures? Grading the whole trajectory, checking each step, something else?

The new Qwen dropped as open weights and our timeline is full of people running it locally. It's fun to watch, and it got us thinking about something we keep going back and forth on.

Everyone calls these models open source, but what actually ships is the weights. You don't get the training data, the exact data mix, or the setup behind the benchmark numbers it launched with. So we can run and fine-tune it, but we can't rebuild it or see how those numbers came together.

Maybe that's fine depending on how you use it. If you just want a strong model on your own hardware, the weights might be all you need. If you're trying to reproduce a result or trust a benchmark, maybe not.

So what has to be open before you'd call a model open source, and not just open weights? Has an open model's published numbers ever landed far from what you saw running it yourself?

Something we keep seeing: more teams are being asked to prove a human supervised an agent. Sometimes it's a security review, sometimes a customer asking. The ask sounds simple. Show that a person looked before the action went through.

Then you open the trace and it gets awkward. There's a row that says approved, or a status field flipped to human-reviewed, but nothing in the run shows a human made a decision. The timestamp could be a default. The reviewer field could be whoever owns the service account. The trace records that a step happened, not that a person was in the loop.

This is an observability problem more than a policy one. If the run never captured the human decision, no policy doc saves you when someone checks. Some of you probably log this cleanly already. Others might open a recent run and find it wouldn't hold up.

So we're curious how you handle it. What do you actually record as proof that a human reviewed something, and has anyone had those logs questioned later?

Midweek thread. Drop whatever you are building, breaking, or staring at.

Most of the bugs that eat our week come from around the model. A retrieved chunk that was quietly wrong. A tool that started returning a new field. A prompt someone tweaked for better results. Every dashboard stays green the whole time.

So what is eating your week right now? Extra curious about the one you were sure was the model, and it turned out to be something upstream.

First time instrumenting an LLM app and not sure what to log? Ask here.

Half the good posts here lately circle the same idea from a few directions. The threads about making the model ask you questions before it answers. The ones about showing it what good looks like instead of describing it. Different framings, one conclusion: the exact words in your prompt are the least important part of it. 

Three things actually move prompt quality, and none of them are phrasing.

Context. The model cannot infer what you never gave it. Plenty of prompts we "fixed" with sharper wording were just missing inputs. Either hand it the material or make it ask for what is missing before it answers.

Contrastive examples. Two or three good-and-bad outputs teach more than a paragraph defining "good." When the same mistake keeps showing up, paste an example of it next to a correct one. The examples do the work your instructions were failing at.

A rubric you can score. Write down what a correct answer has to contain, then run the prompt across ten or twenty real cases and check each against that list, instead of reading one nice-looking output and calling it done.

The part that took us too long to accept: a better-sounding prompt usually scores about the same once you measure it. We would rewrite something, read a single output, decide it felt sharper, and ship it. When we finally scored the reworded version against a batch of saved cases instead of trusting one run, it came out no better than the original, sometimes worse. The examples and the rubric are what moved the number, every time.

Without a definition of good you can measure, you are tuning on vibes. That shift, from wording to scoring, is closer to prompt evaluation than prompt engineering, and it is the part that holds up as the model underneath you keeps changing.

What's a prompt where the wording mattered far less than the examples you fed it?

A new agent framework shows up every couple of weeks, and every other thread turns into LangGraph vs CrewAI vs whatever launched on Tuesday. If you have put any of these in front of real traffic, you know the framework is rarely what decides whether the agent holds up.

Look at what they give you in 2026 and they have converged on the same primitives: a tool-calling loop, memory, streaming, multi-agent delegation, and MCP support. The rest is mostly taste. LangGraph leans on an explicit graph you control node by node. 

CrewAI models agents as a crew with roles and tasks. OpenAI Agents SDK stays lightweight with handoffs and built-in tracing. 

Claude Agent SDK hands you the same harness and subagents that run Claude Code. Pydantic AI gives you type-safe, validated outputs. Google ADK spreads across languages and plugs into Google Cloud. 

Pick the one that matches how you think and move on.

What decides whether it holds up in production sits outside the framework:

• an eval and regression set you trust, so a model swap that breaks last week's behavior shows up before it ships
• step-level tracing, so when a run goes wrong you can see which tool call or handoff did it
• runtime guardrails on the actions that carry consequences
• a memory strategy you set on purpose

None of the six saves you here. An agent that looked fine all week will call the same tool twice and force-push over its own branch. You find that in a trace, and no framework doc will tell you why.

Make the framework call and keep building. Your months go into the eval set, the traces, and the guardrails, because that is what you will be debugging six months from now.

If you have shipped agents on two frameworks, did switching change your reliability, or was it your eval and tracing setup that moved the numbers?

These two terms get used interchangeably and it causes real confusion, so here is the split as we understand it.

Offline evals run against a fixed dataset, usually in CI or before a release. You have expected outputs, or at least a reference. The question they answer is: did this change make things worse than the last known-good version? They are repeatable, they gate merges, and they are the only place where a straight before-and-after comparison is meaningful, because the inputs are held constant.

Online evals run against live production traffic, scoring real requests after they happen. There is usually no reference answer, so the scoring is either a model judging the output, a heuristic check, or a signal from the user. The question they answer is: what is actually happening to real users right now, on inputs nobody thought to put in the test set?

The failure mode of running only offline evals is that your dataset ages into irrelevance while real traffic drifts somewhere else entirely. The failure mode of running only online evals is that you find out about a regression after it has already shipped, and you cannot cleanly attribute it because a hundred other things changed too.

They are not competing. Offline tells you whether to ship. Online tells you what shipping did.

For the online side specifically, the mechanics seem harder to get right: what fraction of traffic do you score, do you sample or score everything, and how do you keep the scoring cost from becoming a line item of its own?

We run LangGraph agents where the model picks tools through structured output. A provider pushed a model update behind the same version alias, and our tool-calls started failing in a way that never surfaced as an error. The API still returned 200. The model still produced text. 

But the arguments for one tool came back as a fenced JSON string instead of a JSON object, and a required field went missing. 

LangGraph's tool node could not parse that, so the tool either did not fire or fired with the wrong input. No exception at the API layer, no alert, just more agent runs doing the wrong thing.

Nothing about a silent swap trips a normal monitor, because the call succeeds. You catch it in one of three places, and the tools you already run draw the line differently. Checked against each tool's current docs:

Tool	Catch before ship (dataset + experiment, gate CI)	Catch in production (online scoring on live traces)	Stop it live (inline guardrail on the response)
LangSmith	Yes	Yes	No
Braintrust	Yes	Yes (async, no added latency)	No
Langfust	Yes	Yes	No
Future AGI	Yes	Yes	Yes

All four give you the same core defense: a fixed dataset of tool-call cases you re-run as an experiment and diff against a known-good baseline, so a format change shows up before you ship. That offline regression set is the part that actually catches a silent swap, and every one of these does it well. 

LangSmith and Langfuse can also gate a deploy in CI on that comparison. 
Braintrust runs its production scoring asynchronously so it adds no latency, which is by design. 

The difference is in the last column. A runtime guardrail inspects the response inline and can block a malformed tool-call before it reaches the user, and among these that is Future AGI's guardrail layer. The others observe and score rather than sit in the request path.

The check that survives a model swap is deterministic, not another model grading the output. For tool-calls, assert the structure directly: parse what the model returned, and for each call require that the name is in the allowed set and that the arguments validate against that tool's JSON schema, exact match on required fields and types.

# deterministic contract test: runs in the eval and in CI, no LLM judge  for call in response.tool_calls:assert call["name"] in ALLOWED_TOOLSjsonschema.validate(call["args"], TOOL_SCHEMAS[call["name"]])

That assertion runs the same way in a scheduled eval and in a CI gate, and it would have caught our swap on the first run, before any user saw it. How are you catching a silent model swap before it hits users?

Midweek thread. Anything goes as long as it is about keeping LLM or agent systems working.

What are you building or fixing right now?

Anything behaving oddly that you have not explained yet?

Anything you tried that turned out to be a dead end? Those are useful to hear about too.

Questions are welcome at any level. If you are just starting to instrument an LLM app and are not sure what to log first, this is a good place to ask.

An LLM guardrail written as a prompt instruction is a suggestion, not a rule. It holds in testing, then quietly stops holding in production, because there is no enforcement boundary. The model is free to generalize around the instruction, and under enough traffic it will.

Concrete version: you tell an agent to never force-push to main. It behaves for weeks. Then a task comes in phrased just differently enough, the context is full of other instructions, and it force-pushes anyway. The rule was always probabilistic.

Three things erode a prompt-level guardrail:

• Instruction competition: every rule you add dilutes the ones already there.
• Context override: later user or tool content outweighs the system prompt.
• Distribution shift: real traffic drifts from what you tested the rule against.

This is not prompt injection. No attacker is involved. The guardrail decays on its own under ordinary traffic, which is what makes it easy to miss.

What holds is enforcement outside the model: a deterministic check on inputs and outputs that can block the action before it runs, plus adversarial testing before you ship.

Which guardrail did you finally move out of the prompt into an enforced layer, and what triggered it?

These two terms get used interchangeably and it causes real confusion, so here is the split as we understandit.

Offline evals run against a fixed dataset, usually in CI or before a release. You have expected outputs, or atleast a reference. The question they answer is: did this change make things worse than the last known-goodversion? They are repeatable, they gate merges, and they are the only place where a straight before-and-aftercomparison is meaningful, because the inputs are held constant.

Online evals run against live production traffic, scoring real requests after they happen. There is usually noreference answer, so the scoring is either a model judging the output, a heuristic check, or a signal from theuser. The question they answer is: what is actually happening to real users right now, on inputs nobodythought to put in the test set?

The failure mode of running only offline evals is that your dataset ages into irrelevance while real traffic driftssomewhere else entirely. The failure mode of running only online evals is that you find out about a regressionafter it has already shipped, and you cannot cleanly attribute it because a hundred other things changed too.

They are not competing. Offline tells you whether to ship. Online tells you what shipping did.

For the online side specifically, the mechanics seem harder to get right: what fraction of traffic do you score,do you sample or score everything, and how do you keep the scoring cost from becoming a line item of itsown?

The loop I keep hitting: the agent calls a tool, the result is not what it expected, so it calls the same tool again with nearly the same arguments. Sometimes it breaks out after a few tries. Sometimes it burns through the step limit doing it, and the user gets a timeout with no explanation.

What I have now is crude. I hash the tool name plus the serialized arguments, keep the last few in the run state, and bail out with a message if the same hash comes up twice. It catches the obvious case.

Two things it does not catch:

Arguments that drift slightly on every attempt. Same intent, different string, different hash, loop continues. This is the common version and my check sails right past it.

Legitimate polling. If a tool is supposed to be called repeatedly until something completes, my check trips on correct behaviour and I have to special-case it, which means maintaining a list.

I have thought about comparing embeddings of the arguments instead of hashing, but that feels like a lot of machinery for a guard rail.

What are you actually running for this? Step budget only, semantic similarity between calls, per-tool call limits, something else entirely?

Unpopular around release week, but here it is. Most of the agent behaviour I have had to fix was not the model being dumb. It was the scaffolding: how tools were described, what got put back into context after a failure, how many steps were allowed, what happened on a timeout.

The test I use now is to debug on a weaker model on purpose. If the flow only works on the best available model, what I have is not a working agent, it is a model compensating for my plumbing. When the frontier model is the only thing holding the loop together, the next behaviour change in that model is going to break me and I will have no idea why.

The uncomfortable part is that harness work is boring. Nobody writes a launch post about tool descriptions that do not overlap, or about the retry policy. But that is where the wins were.

I am not claiming model quality does not matter. It obviously does, and there are tasks that simply do not work below a certain capability. I am claiming the ratio is nothing like what the discourse suggests.

Where do you think the line actually sits? Curious if anyone has the opposite experience, where swapping the model fixed something the harness could not.

LLM-as-a-judge gets used as if it is one thing, so here is a plain teardown of the mechanism and where it goes wrong, for anyone leaning on it to score their outputs.

The mechanism. You take a model output, hand it to a second model along with a rubric, and ask that model to score it. There are two common shapes. Pointwise: show one answer, ask for a score against criteria (the G-Eval line of work does this, and asks the judge to reason through the criteria before scoring). Pairwise: show two answers, ask which is better. Pairwise tends to be more stable because “A is better than B” is an easier, more consistent judgment than “this is a 7 out of 10”.

The biases, all documented in the research on judge models:

• Position bias. The judge favors whichever answer is shown first, regardless of quality. Mitigation: run both orders and average, or swap and check for agreement.
• Verbosity bias. The judge favors longer, more elaborate answers even when they are not more correct. Mitigation: control for length, and watch whether score tracks word count.
• Self-enhancement bias. A judge tends to rate outputs from its own model family more highly. Mitigation: be careful using the same model to generate and to judge.

None of this means judges are useless. It means a judge is an instrument with known error modes, and you should calibrate it against a set of human labels before you trust its numbers, then re-check periodically.

How are you keeping your judge honest? Anyone regularly measuring judge-to-human agreement rather than assuming it holds?

A while back we open-sourced most of our agent stack, tracing, evals, guardrails, simulations, a gateway, under a permissive license so anyone could self-host it on their own infra. Putting the code on GitHub is the part every "why we went open source" post already covers. The part that reshaped how we build came after, once strangers started running it on machines we will never see.

Here is the gap nobody warns you about. On our own hardware the thing runs because the environment is ours. The model endpoints are reachable, the keys are already set, the versions match what we tested, and a hundred small assumptions hold without anyone deciding they should. None of that is written down, because it never had to be. Then someone clones the repo onto a setup that shares none of those assumptions, and every one of them turns into a question in the issue tracker.

The useful part was which assumptions surfaced, and it was always the boring ones. A path that only existed on our boxes. A model we had assumed everyone could reach. A default that made sense in our region and nowhere else. A setup step that everyone here knew to run and no one had thought to document. You cannot find these from the inside, because from the inside they are invisible by definition. It takes a stranger on unfamiliar infra to trip over one, and once someone hits it in public, you fix it for good.

That reordered what we worked on. Before, we prioritized by internal conviction, the features we were sure mattered. After, the issues that moved the roadmap most were the plain "this will not run on my stack" reports. None of those showed up in a planning meeting, because the person hitting them was never in the room. Each one was a person who wanted to use the thing and could not, which counted for more than any feature we had lined up, and the public backlog slowly became the real roadmap.

The lesson that stuck is that a permissive license is the easy 10%. Anyone can push code and call it open. The other 90% is making it survive contact with an environment you have never seen and cannot test for, and that work only starts once people run it somewhere you don't control. You end up debugging a machine you have never logged into, described by someone who has no reason yet to trust you. It is also the only reason the project became portable in practice instead of portable on a slide.

So for anyone who has open-sourced something meant to run on other people's infra: what assumption baked into your own setup was the first to break when a stranger ran it? And did going public end up reordering your roadmap the way it did ours? 

Most discussion about agents fixates on the model or the framework. The choice that quietly shapes how an agent behaves gets skipped over: where the control flow actually lives. For a lot of agents built today, every branch, every role, and every stop condition sits inside one system prompt doing all the work.

That single-prompt setup is the standard agent loop. One prompt instructs the model to reason about the task, pick a tool call, read the result, then decide what to do next, over and over until it judges the job done. The same prompt holds the orchestration logic, the persona for each sub-task, the formatting rules, and the exit criteria. Each tool result gets appended into the same context window, so the input grows with every step. Nothing about which path the agent takes is written down anywhere except as instructions in that prompt. 

This holds up until it doesn't. As the tool count climbs, the prompt has to describe all of them, and a single system prompt crossing 30k tokens is not unusual. Tool selection turns non-deterministic: the same request takes a different path across runs for reasons the prompt can't pin down. Debugging agents built this way is hard because there is no isolated step to inspect, only the whole loop replaying against a different context each time. People report the same input producing a different tool call dozens of times with no way to reproduce it.

Two things change when the control flow moves into code:

The branching becomes a graph of nodes and edges, closer to a state machine than a block of prose. Each node gets its own small prompt with one job. A routing node only classifies intent and returns one label. A node that drafts a reply only drafts. These prompts are short, their outputs are narrow, and each one can be tested on its own with fixed inputs.

State stops living in the transcript. Instead of the model inferring progress from a growing pile of appended observations, state becomes an explicit object that each node reads and updates, and the edges decide what runs next. The path through a multi-step run is defined in code rather than implied by a paragraph. Recovery gets cleaner: since each step is a discrete node with saved state, a failed step can be retried or resumed from that point instead of replaying from the first token.

None of this makes the model better, only easier to see what the agent is doing. Curious where others draw the line: at what point did moving control flow out of the prompt start paying off for your agents?

hey developers, write about what you're working on without making a regular post.

Three things worth answering, take whichever applies:

• What are you building or shipping this week?
• What is blocking you right now?
• What would you like a second opinion on before you commit to it?

Rough is fine. Half finished is fine. A question you have been sitting on for a fortnight is very fine.

We read every reply in this thread and we answer all of them. If you post and it looks like nobody is around, we are around.

If you are new here this is a good first place to say hello, and you do not need anything impressive to report. “Trying to get tracing working and losing” is a perfectly good answer, and honestly it tends to get better replies than a success story does.

Most posts about agents getting stuck in a loop stop at the symptom. The run gets cut off, someone raises the step limit, and the same agent runs longer before failing the same way. The cap is the thing that ends the run, so it gets treated as the thing to tune.

A loop forms for one of three reasons, and none of them is the cap. The agent has no record of what it already tried, so a similar state produces the same reasoning and the same call. Or the tool returns prose it cannot read as done or failed, so calling again is the safer guess. Or nothing checks whether the goal is met, which leaves the cap as the only thing that ever ends the run.

A cap only guarantees the run ends. It says nothing about whether the work got done, so a run that stops at the cap looks the same whether it finished the job or never got close. Raising the number buys a longer and more expensive version of the same failure. LangGraph's own error ends with "reached without hitting a stop condition," and on 1.2.9 the recursion_limit that triggers it ships at 10007, so the default is not saving anyone. The OpenAI Agents SDK is tighter, max_turns defaults to 10 and it raises MaxTurnsExceeded.

The simplest way to check is to hash each call into a brief ID, such as a fingerprint, before it executes. Rough shape:

import hashlib, json

seen = set()

def action_key(tool_name, args):
    blob = tool_name + json.dumps(args, sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()

def guard(tool_name, args):
    key = action_key(tool_name, args)
    if key in seen:
        return "REPEAT_BLOCKED: this exact call already ran"
    seen.add(key)
    return None  # allowed

Hash the tool name and arguments, keep the keys for one run, and check the set before dispatching. A hit means the agent is about to redo work it already did, which you can block with a note back to the model or treat as a stop signal.

sort_keys=True is not cosmetic. Without it, the same call with arguments in a different order hashes to a different key and passes as new work, and models do not emit arguments in a stable order. On Python 3.13, {"q": 1, "db": "x"} and {"db": "x", "q": 1} collapse to one fingerprint.

Two things it will not catch. Any volatile field, a timestamp or a request id, gives every call a fresh fingerprint while the agent goes nowhere, so strip those before hashing. And it only watches the call side, so two different calls that keep returning the same dead-end result never trip it.

For anyone running agents long enough to hit this: does fingerprinting the call catch most of your loops, or did you end up having to fingerprint what came back?