u/Future_AGI

If you have ever tuned a prompt by hand, you already know the pattern.

You make a small change, run the same examples again, and hope the output gets better without breaking something else. Sometimes it works. Sometimes it gets worse in a way that is hard to spot until later.

That is the problem we wanted to make more structured.

We built prompt optimization in-house and shipped it as an Apache 2.0 Python library so people can move from manual prompt edits to a repeatable improvement loop.

The idea is simple: take a prompt, run it on real data, score it with evals, and let the optimizer search for better versions instead of guessing by hand.

We support 6 optimization algorithms:

• GEPA
• PromptWizard
• ProTeGi
• Bayesian Search
• Meta-Prompt
• Random Search

Why 6? Because different prompts behave differently.

Some prompts need a search strategy that explores more. Some work better when the optimizer changes the wording in a more guided way. Some need a judge signal that is very clear and task-specific. In practice, the “best” optimizer depends on your data, your evals, and how messy the task is.

This is built for people who are actually shipping prompts, not just experimenting with them in notebooks.

If you are working on RAG, support flows, extraction, copilots, or any system where prompt quality changes the outcome in a measurable way, the goal is the same: make improvement repeatable instead of manual.

A typical run looks like this:

• Start with a baseline prompt.
• Run it against a dataset.
• Score the outputs with your evals.
• Generate candidate prompts with an optimizer.
• Compare the results.
• Keep the version that performs best.
• Repeat when your data changes.

What we have found is that prompt work gets much easier once the loop is clear. You stop asking, “Which wording feels better?” and start asking, “Which version actually performs better on the cases that matter?”

That is what we wanted to build.

The open-source platform for shipping self-improving AI agents. Evaluations, tracing, simulations, guardrails, gateway, optimization. Everything runs on one platform and one feedback loop, from first prototype to live deployment.

Who is this for?

• Prompt engineers who want a repeatable optimization flow.
• Builders shipping production prompts who need safer iteration.
• Teams comparing different optimization methods on the same dataset.
• Anyone who wants prompt quality to be measurable instead of subjective.

What can you do with it?

• Optimize prompts with six different algorithms in one library.
• Run a prompt against a dataset and compare candidates side by side.
• Use your own evals to define what “better” means.
• Keep optimization tied to real task performance.
• Move from one-off edits to a loop you can actually reuse.

If you are working on any project with prompts, try it in your own workflow and see what the optimizer changes. It is open source, and you can also layer it with other open-source tools for evals, tracing, or simulation if that fits your setup.

If you ask most teams “do you trust your agent in production?”, you usually get a shrug and a story, not an answer. Actually we get the same answer

Dashboards, a few example chats, maybe a one-off eval notebook… but very few people can point to a clear, living eval setup and say: “this is why we still trust it today, not just the week we shipped it.” honestly.

We have spent the last 18 months talking to teams running agents for support, internal copilots, RAG search, and multi-step workflows, the same problems keep coming up.

• When something goes wrong, it is hard to tell which step actually failed.
• Retrieval quality drifts, but there is no way to tie a bad answer to a specific tool call or document.
• Eval sets are written once and slowly rot while prompts, tools, and models keep changing.
• Real failures in production rarely make it back into the test set, so the system keeps “passing” old tests.

At that point, saying “the agent is in production” does not mean “we understand its behavior.” It mostly means “nothing has burned down yet.”

The way we started thinking about it is simple: if agents are systems, not single prompts, then “evaluation” has to follow the system, not just the final answer.

If agents are systems, not single prompts, then “evaluation” has to cover more than final answers.

we think a serious agent stack needs at least four things:

1. Tracing down to the step level, so you can say “step 4 failed because retrieval returned garbage” instead of “the agent was bad here.”
2. Evaluations that can be tied to tasks and steps, not just global thumbs up or down.
3. Simulation so you can test agents against a wide range of scenarios before users discover the weird edge cases for you.
4. A feedback loop where production failures become new eval cases, so the system does not just keep re-passing the same old test.

We ended up building our own stack around that idea and then open-sourcing it.

The open-source platform for shipping self-improving AI agents. Evaluations, tracing, simulations, guardrails, gateway, optimization. Everything runs on one platform and one feedback loop, from first prototype to live deployment.

Who is it for?

• People building agents, copilots, and RAG systems who want to see where the system actually fails, not just whether it “looks good” in a few test prompts.
• Teams who want to keep eval logic and traces inside their own stack instead of pushing everything into a closed SaaS.
• Anyone who wants to treat agents as systems to monitor and improve, not features to “fire and forget.”

What can you actually do with it?

• Trace every call, tool use, and step in an agent flow, with enough detail to debug real failures.
• Run evaluations with readable scoring code that you can change when your domain needs different rules.
• Generate and run simulations so you can see how the system behaves under varied, messy inputs.
• Close the loop by using eval results and traces to drive fixes, guardrails, and optimization.

We have open-sourced the same stack we run ourselves, and the repo has now crossed 950+ stars with people starting to use it and push on it in real projects.

The reason we are sharing it here is less “launch” and more “sanity check.”

If you think about agents and evaluation seriously, what do you see as missing from most stacks right now?

Is it better task-level metrics, better traces, better simulation, a cleaner feedback loop from production, or something else entirely?

If you want to try what we built in your own setup, the links are in the first comment.

A few weeks ago, we shared Future AGI here as our open-source AI stack for production agents.

Since then, the project crossed 800+ GitHub stars, people started contributing, and the feedback got much more real.

The useful part was not the launch itself. It was seeing what happened once developers started trying to use the stack in their own workflows.

Some people came in through tracing. Some cared more about evals, simulations, or guardrails. Some wanted the full loop, from prototype to production, without stitching five separate tools together.

That has been the most interesting part for us.

The open-source platform for shipping self-improving AI agents. Evaluations, tracing, simulations, guardrails, gateway, optimization. Everything runs on one platform and one feedback loop, from first prototype to live deployment.

That sounds clean on paper. Open-source gets honest very quickly once people try it in real projects.

If setup is rough, people notice. If the docs miss a step, people notice. If a workflow makes sense in theory but feels awkward in practice, people notice.

That has helped a lot.

It has pushed us to think less about what sounds good in a launch post, and more about what actually helps a developer once an agent starts failing in non-obvious ways.

A few parts of the stack seem to pull the most attention:

• traceAI, when teams want visibility into model calls, tool calls, latency, and failures.
• evaluations, when teams want something more concrete than “the output looked fine.”
• simulations, when teams want to test behavior before production becomes the test environment.
• the broader loop, when teams want tracing, evals, guardrails, gateway, and optimization to work together instead of living in separate dashboards.

Once developers start using a stack in real agent workflows, the truth shows up fast.

That is where the rough edges become obvious, setup gaps, broken assumptions, missing steps, workflow friction, and bugs that no launch post will catch. If you are building with agents, try it in your own flow, build something with it, and tell us where it breaks or feels harder than it should.

That kind of feedback is the most useful one for us right now. What worked, what did not, what felt confusing, and what you would want fixed before trusting it in a real system.

If you have not tried it yet and want to explore it, the links are in the first comment.

Once an agent is live, the next question gets surprisingly hard to answer.

How do you know it is actually working?

Not in a demo. Not on a benchmark. In production.

We have spent a lot of time looking at agent systems across support bots, internal copilots, RAG workflows, and multi-step setups.

The surprising part is that the model is usually not the main problem. The harder part is defining what “working” means, then measuring it in a way that survives real usage.

A few patterns keep coming up.

An “autonomous research agent” gets judged by thumbs-up rate, but nobody can clearly describe what the bad 20% actually looks like.

A multi-agent workflow fails, but the team cannot tell whether the issue came from retrieval, routing, tool use, or state passed between steps.

An eval set looks strong in staging, but nobody is measuring production outputs closely enough to know whether that behavior holds up under real traffic.

A team says the agent does the job well, but they have never run it enough times across varied inputs to know where consistency starts to break.

That has changed how we think about production agents.

The fix is usually not “switch to a better model.” More often it is one of a few less glamorous things.

Write down what success looks like in a form that can actually be graded.

Trace each step, not just the final output.

Run broader scenario coverage before production sees the edge cases first.

Take failures from production and push them back into the eval set so the system does not keep passing the same stale checks.

That last one feels especially important.

A lot of eval sets get written once, then stay mostly frozen while prompts, models, tools, and workflows keep changing underneath them.

But the issue is that many teams still talk about agents like they are features, when in practice they behave more like systems.

They have state, dependencies, failure modes, and weird interactions between parts. If that system is non-deterministic, then the job is not to pretend it is deterministic. The job is to make the behavior visible enough to debug, score, and improve.

That is where evals and observability start to matter.

Not as reporting layers, but as the thing that makes non-deterministic behavior legible.

We are curious how this looks for others shipping real agents.

What was the first thing that broke once your agent hit real users?

Hi community.

I am Nikhil, the founder at Future AGI, we build an open-source platform for evaluating, observing & guardrailing AI agents. Posting because 6 months in, I keep noticing the same pattern across teams and want to know if it's real. Not here for signups, demos, links, or DMs, bio has the rest. What I'd love: honest pushback, especially from people who think I'm wrong.

Now the actual thing.

I thought the hard part of building this would be the engineering, wrangling LLM outputs, building scoring, making dashboards fast. That part is work, but it's tractable.

The actual hard part has been getting teams to admit out loud what "good" means for their AI.

Stuff we keep watching happen:

• A team ships an AI feature. Three months in I ask "how do you know it's working?" The answer is some variation of: thumbs up/down counts, vibe checks from the PM, or "nobody's complained loudly enough." That's the whole evaluation.
• Eval criteria written down on a whiteboard during kickoff. Three weeks later, half of them contradict each other once there are real outputs to grade. Nobody had forced themselves to think it through.
• The "we need evals" request comes from engineering. The "we already have evals" pushback comes from leadership. They mean completely different things by "evals." Nobody has named the gap.
• Teams will spend a month building a fancy eval pipeline before deciding on one metric they actually care about. The pipeline runs. Numbers go up. Nobody knows if that's good.

What I keep getting wrong as a builder:

1. I over-invest in better measurement before the customer has decided what to measure. Slow lesson: a clear definition of "good" beats a sophisticated evaluator of "??".
2. I assume people will read a 600-word doc explaining a methodology. They will not. They will read a screenshot of one specific failing example.
3. I keep building features for the eval pipeline. What customers actually want is the conversation that forces their team to align on what good looks like. The pipeline is just the excuse for the conversation.

Broader pattern I think we're seeing: most AI products are running blind, not because the tooling doesn't exist, but because nobody on the team has had the uncomfortable conversation about what failure even looks like. The tool is downstream of that conversation.

The version of this that actually works, I think, is a boring closed loop: catch what breaks in production, know why it broke, fix it, then ship a better baseline next time. That's what "self-improving agents" actually means in practice a loop that closes. Without that, it's three new dashboards and a Slack channel nobody opens.

So here's the honest ask, since I said I'd be transparent about it:

I want to know if the pattern is real for you, or if I am building for a problem only a small slice of teams actually have.

If you're shipping AI features in any form internal copilot, customer-facing chat, agent, RAG, anything three questions I'd genuinely love an answer to:

• What does "good" actually mean for your AI, in a way that survived contact with real model outputs?
• How are you catching regressions right now? Structured evals, golden datasets, prod logs, thumbs-up/down, or honestly just gut feel?
• Where does your current approach hurt the most?

If your honest answer is "we don't really have one yet" please say that. That's the most useful data point I could get.

And if you think the whole framing is wrong, tell me why. That's the comment I'm hoping for most.

The same question lands on this sub a few times a week, and the standard answers (RAGAS, DeepEval) are correct but stop one layer short of what you actually need once your app leaves a notebook. Wanted to lay out the full picture for anyone learning this in 2026.

LLM evaluation tooling sits in three layers. Most learners get pointed at layer one, hit a wall, and assume the field has nothing else to offer. It does.      
  
Layer 1: Metric libraries                                                                                                                                      

RAGAS is the cleanest example. You hand it rows of (question, context, answer, ground truth) and it scores each row on faithfulness, answer relevancy, context precision/recall, noise sensitivity, plus newer agentic metrics (tool call accuracy, agent goal accuracy).

Good for: a static eval set, an offline notebook, a paper.                                                                                                 

Limit: shaped around RAG. Once your app is an agent loop or multimodal beyond images, the metric set thins out fast.

Layer 2: Test frameworks
DeepEval is the canonical one. ~50 metrics including G-Eval, hallucination, bias, toxicity, task completion, tool correctness, plus image-level metrics. Pytest-style assertions, CI hook, custom LLM-as-judge.

Good for: regression-testing prompts and chains the way you regression-test code.
Limit: mostly offline. It tells you version N+1 is worse than N on a frozen dataset. It will not tell you what is happening on real traffic at 3 AM, or which  span in a 20-step agent trace produced the failure.                                                                                                            
  
Layer 3: Observability and evaluation platforms                                                                                                                

The layer most tutorials skip, and the layer most production teams end up at. Tools here include Arize Phoenix, Langfuse, Braintrust, and Future AGI's ai-evaluation. They sit on top of OpenTelemetry traces (the GenAI Semantic Conventions are now a real spec) and run evaluators against live spans, not only static datasets.                                                                                                                                               

One technical detail worth knowing about this tier: almost all of them call third-party LLM judges (GPT-4, Claude) under the hood, so eval cost scales linearly with traffic and you inherit the judge model's latency. The interesting outlier is ai-evaluation, which ships its own trained evaluation models (the TURING family, covering text, image, and audio) and runs guardrails sub-100ms on live spans. 

Different trade-off: fixed-cost, low-latency scoring vs. the flexibility of swapping judge models per metric. Whether it matters depends on your scale, an MVP doesn't care, an app doing online evals on every request very much does.
  
Good for: real users, agent loops, multimodal inputs, drift over time.                                                                                         
Limit: heavier setup. You instrument your app and accept some vendor coupling.

Why this matters more in 2026
Agents are now the default architecture. A single query can fan out into 20+ LLM calls, tool invocations, and retrieval steps. Sierra Research's τ²-bench (2025) showed dual-control settings cause large drops vs. single-turn evals; SWE-bench Pro pushed top models to ~23% from 70%+ on Verified. A single faithfulness score on the final answer hides where the failure happened.                                                                                       

Multimodal is also in production. lmms-eval v0.5 added 50+ audio/vision benchmarks; Video-MME (CVPR 2025) is the de facto video MLLM benchmark. The metric libraries have not caught up, and only a couple of the platform-tier tools natively score audio or video today.

A rough decision rule

-Static RAG dataset, offline only: RAGAS.                                                                                                                     
-Prompt or chain regression in CI: DeepEval or promptfoo.
-Production traffic, agents, multimodal, drift: a platform-tier tool.                                                                                 -All three together is normal. They compose.   

Question for the sub
For anyone running LLM apps close to or in production: what single metric has actually caught regressions for you, and how often does your judge disagree with your own review when you spot-check? Curious whether anyone has wired their CI eval into a production observability tool, and what the integration pain points were.                                                                                                                                                          

Happy to go deeper on any layer in the comments. 

Software testing has a quiet assumption underneath it. Run the same input twice, get the same output twice.

In early day of building most of our tooling and instincts are built on top of that one line.

Agents break that assumption immediately.

The same prompt, the same tool definitions, the same user message can produce a different sequence of tool calls depending on the agent's "mood" (yes, that is a real variable now), the temperature setting, or whether the user said "please".

We started by trying to force agents into our existing test framework. Pin the seed, snapshot the outputs, diff the trees.

None of it worked, because the bugs were not at the level we were testing.

The actual production failures were sequencing bugs. The agent calling tools in the wrong order under pressure, skipping a verification step when a user got emotional, or calling the same tool twice and getting confused about its own state.

You cannot find these by reading transcripts, because there are too many. You cannot find them with assertions, because there are too many valid sequences. You cannot find them with pinned seeds, because the failure mode is the variance itself.

What started working for us was treating the agent less like a function and more like a service we have to load-test continuously.

We started calling this internally a self-improving AI agent pipeline, because the agent's production failures become inputs to its next version, with no human in the middle of the loop.

The loop has three parts.

Generate synthetic users with explicit personas (mood, context, pressure tactics) and drive them at the live agent.

Score the transcripts against domain metrics. Function-call accuracy and ordering matter much more than output similarity.

Take the failures and feed them back into prompt rewrites or fine-tuning data. Run the simulation again.

It looks more like fuzzing than unit testing.

The point is not to catch one specific bug. It is to surface the distribution of behaviors and shrink the tail of the bad ones.

The thing we did not expect was that the prompts that came out of this loop were uglier than what we would have written by hand. Long, defensive, full of clauses our reviewer would have flagged as redundant.

They were not redundant. They were closing failure modes a human did not think to script.

We are starting to think the SDLC for agents is going to look closer to chaos engineering than to traditional testing. Less "does this assertion hold" and more "how often does the whole system stay inside its envelope across a population of users".

The question we keep getting stuck on: how do you decide when to stop iterating, since this kind of loop technically never converges?

If anyone wants the full breakdown of how the self-improving AI agent pipeline works end-to-end (architecture, evals, the optimizer we use, code samples), link is in the first comment.

We've been going deep on evaluation pipelines lately and wanted to share something that clicked for us the difference between LLM Judges and Eval Agents, and why you probably need both.

Most of us start with LLM-as-a-Judge. You write some instructions, throw in your {{input}} and {{output}}, pick a model, and get a Pass/Fail back. It works, it's fast, it's cheap. For things like relevance, tone, coherence, honestly it's pretty solid.

But here's where it breaks down.

Say you're evaluating a RAG pipeline and you want to know if the generated answer is actually factually correct based on your internal docs. An LLM Judge can't do that, it can only judge what's in the prompt. It has no way to go verify anything.

That's where Eval Agents come in.

Instead of a single-turn judgment, an Eval Agent actually reasons over multiple steps, up to 15 iterations and has access to tools like knowledge base search, web browsing, or external APIs. So it'll go look something up, compare it, reason about it, and then give you a verdict with a detailed explanation.

Our rough mental model for when to use each:

• Checking if a response is helpful, on-topic, or well-written → LLM Judge (fast + cheap)
• Verifying facts against your docs or the web → Eval Agent
• High-volume batch scoring → LLM Judge
• Auditing complex multi-step agent traces → Eval Agent
• Format/regex/exact match checks → Code Eval (don't burn LLM tokens on this)

The trap we see a lot of devs fall into is trying to make LLM Judges do everything. They end up writing these massive prompt instructions trying to get the judge to "simulate" fact-checking, and it just doesn't hold up at scale.

Use the right tool for the job. LLM Judges for subjective quality. Eval Agents for anything needing real reasoning or external grounding. Code Evals for deterministic stuff.

we are just want to know what evaluation setups you all are running in prod are you doing anything beyond basic LLM Judges?