How do you gate what your agents are actually allowed to do in prod?
In our company we tried to send emails and touch Stripe with the agent but it feels not really save.
Our first version was just if/else in the tool wrapper. Allow Stripe, deny refunds over some amount. It worked for a short time but after defining multiple tool definitions it was kinda messy and we needed a separate policy layer while also putting a human in the loop when the policy definitions are not enough.
What we really want is to block the API call while a person looks at the tool the payload and hits approve or reject. For now we handled that with automated Slack messages, but ther're slow to respond, and it always took them about 40 minutes to process a request for us.
We also don't really like handing the agent a long-living API key, which in hindsight is one prompt injection away from a very bad day...
So I'm curious how you currently solve these problems:
- Policy as code, or still hardcoded in the tool layer? Anyone using OPA for this or is that overkill?
- Has anyone made in-line human approval not feel terrible?
- Do you scope credentials per action, or is everyone still passing the real key?
Disclosure so it's not weird later: I'm building in this space. Not pitching it here. I mostly want to know if everyone else solved this and I missed it.