Can an AI agent legitimately govern another AI agent?
I've been seeing more architectures where one AI agent carries out tasks while another monitors its behavior, evaluates risks, and can stop actions from proceeding. That can certainly be useful, but there's something I keep coming back to. If the monitoring agent is also relying on a model to decide what should or shouldn't happen, where does the actual governance authority come from? Are we introducing independent oversight, or just asking another model to make the call?
It gets even more interesting when an action is put on hold. If that hold can be lifted, who or what has the authority to release it? If it's a human, having permission to approve something doesn't automatically establish that they're qualified for that particular decision, sufficiently independent, or still authorized to make it. And if the decision goes to another AI agent instead, we haven't really answered the original question. We've just moved it somewhere else.
I don't see a problem with AI helping to monitor systems, identify risks, or support human judgment. Those are useful roles, even if their assessments remain fallible. What I'm struggling with is the point where an AI-generated assessment becomes a binding governance decision. What actually gives that determination authority, and who gets to establish that the authority is valid at the moment it matters?