How are you authorizing AI agents to take real-world actions?
Spent last week debugging something that technically wasn't a bug. Had a support agent (LangGraph, calling a refunds endpoint) that did exactly what it was built to do: valid API key, right scope, hit the right route, well-formed request. It also issued a refund that should never have gone through — no prompt injection, nothing exotic, just a normal conversation that ended somewhere it shouldn't have.
There's nowhere in the stack that would've caught this. The API key answers "is this service allowed to call /refunds." It doesn't have an opinion on whether this specific refund should happen. The only thing telling the model not to do that is the system prompt, and a system prompt isn't a control, it's a suggestion the model is statistically likely to follow most of the time.
Tried a few things to fix this properly and keep hitting walls. A config file with thresholds means engineering's still in the loop for every change. A DB lookup before each action adds a round trip you may not want on the hot path. And the second you need any real logic — different limit for this customer segment, different rule during an incident window — you're basically writing a rules engine, badly, inside a middleware file, instead of admitting that's what's needed.
Haven't found a clean writeup of anyone solving this well. OPA and Cedar exist but nobody seems to actually want to hand-write policy in either past a proof of concept — the syntax is its own thing to learn.
If you're running agents against anything with real consequences like payments - how are you actually enforcing limits at the point of execution? Hardcoded if-statement near the tool call? An actual policy layer? Hoping the prompt holds? And if there's a real policy layer, who's allowed to change the rules without opening a PR — engineering only, or does support/compliance have a way in too?