Managing AI agent identities in production, how are you all keeping it sane?
Hi,
We have a small platform where different agents handle different things for the same customer account. Eg, a support agent that talks to Zendesk, a billing agent for Stripe, one "ops" agent that talks to some internal APIs, plus a generic assistant agent that shows up in the app.
Internally they're all just LLM backed workers with tools, but they each have slightly different behaviour, prompts, access scopes, logging rules etc. Right now the way we identify them is kind of hacked together. We have agent ids in config, some names in the UI, some stuff in headers like x agent name, and then per tenant we sometimes clone configs because someone wants their "assistant" agent to behave more like support.
Its already getting messy In logs, you see a mix of agent id, display name, and whatever the caller put in. In our metrics, we have tags like agent type, agent role, agent namespace. Then routing decisions depend on a mix of user intent and these tags. Feels like we're piling identity onto identity and slowly losing track of what each agent is.
What im trying to clean up rn is:
Central naming / namespacing so an agent has one canonical identity
Something like agent classes vs instances so we can say "this is a support agent" with variants per tenant without duplicating logic everywhere
Clear separation between "who the user thinks they are talking to" and "which internal worker actually handled the request"
I keep bumping into tiny questions all over. Eg, when we spin up a new agent just for a one off workflow, do we give it a durable id and log it forever, or treat it as ephemeral. When we retire an agent, what happens to historical data and dashboards. When an agent calls another agent, which identity shows in the audit trail.
We also need some way to express permissions cleanly, like "support agent can call tickets api but not payments", without having permission sprawl per instance. Right now its half in code, half in config, and I lowkey hate it.
If you're running more than one agent in prod, how do you model identities and roles so it doesnt turn into a naming dumpster fire after a few months? Any tips or patterns appreciated, idk if I should lean more on something like service accounts, or treat agents closer to users in the system.
thanks in advance.