u/CommonSuch4138

Been running a few agents in production for a couple months now. Nothing crazy, but enough that I'm spending way too much time clicking through traces when something breaks.

Currently just using basic logging + Langfuse for traces. It works, but I feel like I'm playing detective every time a user says "the agent gave me a weird answer." I find the trace, click through 20 spans, cross-reference with tool logs, and 45 minutes later realize the issue started 5 steps before the error.

What's your actual workflow when an agent fails in production? Are you just manually digging through traces too, or am I missing something obvious?

Also how do you handle the "slow degradation" stuff? No errors, everything green, but outputs just... drift?

Been running a few agents in production for a couple months now. Nothing crazy, but enough that I'm spending way too much time clicking through traces when something breaks.

Currently just using basic logging + Langfuse for traces. It works, but I feel like I'm playing detective every time a user says "the agent gave me a weird answer." I find the trace, click through 20 spans, cross-reference with tool logs, and 45 minutes later realize the issue started 5 steps before the error.

What's your actual workflow when an agent fails in production? Are you just manually digging through traces too, or am I missing something obvious?

Also how do you handle the "slow degradation" stuff? No errors, everything green, but outputs just... drift?

So, am a software engineer by profession and like to build some side projects on my weekends and have been trying to build some ai agent, like recently i build an agent for BTC up-down trading, sometimes it works well and sometime dont, the main problem I face is when i leave it to work on its own their is fuck up everytime and i dont know what actully broke, so tried to learn about it and got to know about Langsmith an ai agent observability tool for agents made on langchain, so this made me curious and i dug deep in this space and apparently its a market of its own, langsmith, langfuse, maxim, braintrust, galileo ai (acquired by cisco), and many more, like i had no idea their will be a space just to keep an eye on the agent and how they are working, I want to know how many of you guys actually use such tools or are these made specifically for enterprise, and would you as an solo dev be interested in using such tool?

This made me wonder, will this space have any chance for any small players to build an observability layer for debugging of ai agent, like what do you guys think should i try to make a similar tool in this space, will their be any opportunity?

[removed]