Has anyone put an AI agent on real production alerts yet? What broke first?
This question keeps coming up with ops teams in Pune: "can we put an AI agent on our alerts?" In my experience, most teams haven't worked out two things before trying. One is what it costs at their alert volume. The other is what happens when the agent is wrong.
A small group of SRE, DevOps and platform folks is getting together in Pune this Friday to work through exactly this, hands-on. Everyone gets their own Kubernetes cluster. We run four AI models on the same ops problem, run the agent with memory on and then off, and then try to break it before it breaks prod.
Before Friday, I'd like to hear from people who've already tried it:
- What did you build yourselves, and what did you take off the shelf?
- How do you price it? Per alert, per incident, or did the token bill surprise you?
- What guardrails did you put in before the agent could touch prod?
- What broke first?
I'll post what comes out of Friday here afterwards.
(Disclosure: I work at NudgeBee, a Pune AIOps company that's partnering on Friday's session. Happy to share details in the comments if anyone wants to join.)