r/AI_Governance

Every enterprise is currently spinning up long PDF documents, compliance checklists, and ethics boards. Everyone is checking the box.

But out in the wild? It’s a total mess.

There is a massive, dangerous gap between AI Policy (what’s written on paper) and Runtime Enforcement (what the AI actually does in real-time).

Here is what’s actually happening under the hood right now:

• Static rules vs. Autonomous agents: Compliance writes rules for humans, but dynamic AI agents don't read PDFs. They connect to production DBs, pull sensitive logs, and execute workflows without understanding what information is actually off-limits.
• Post-mortem auditing is not protection: Most security tools today just tell you what went wrong after it happened. That’s not enforcement; that’s just a digital autopsy. Monitoring a code leak or a bad API call after the fact is already too late.
• The missing execution layer: There is almost zero real-time blocking or human-in-the-loop validation built into the execution layer. If an agent gets a bad prompt or hallucinates a command, nothing stops it from executing.

Writing an AI policy doesn’t mean you have AI governance. If you can’t enforce it at the execution level, you don't have control.

Curious to know how other teams are tackling this, who in your organization actually owns the real-time enforcement of AI agents? CEO /DPO / CISO ??

Business is exploring democratizing AI solutions instead of trying to manage pockets of user-managed/deployed / non-enterprise tools. We've had a chat with OpenAI and Microsoft already, with Anthropic coming next. It looks like, despite Copilot's less than stellar reputation, Microsoft's AI stack seems to be far ahead of the others from a management/governance perspective, especially for a M365 shop (like we are, heavy SharePoint usage).

Is that the same experience / feeling some of you have had as well? Thoughts?

we’re building out some AI features for our app, things like chat responses and recommendations. mostly using gpt4o with some fine-tuning, expecting around 10k users once it’s live.

rn we rely on basic output tests and some manual reviews, but it’s slow and doesn’t cover edge cases well.

we tried adding tracing and eval tooling, but setup and maintenance ended up taking more time than expected. integration into our workflow has been the bigger issue than the tools themselves.

pressure from product to move faster, but our last beta surfaced a few hallucinations that almost made it to production. trying to find a way to validate behavior more consistently without turning it into a full-time effort.

what approaches have worked for you in catching issues early without slowing things down too much?

Hey everyone this is my first time posting so please bear with me. This is not a self-promotion rather needed some advice.

I’m working with a pre-seed startup. We’re building a governance layer for fintech companies deploying AI models and agents in regulated workflows.

The product combines:

- A runtime governance layer that sits around AI models and agents, checking inputs, outputs, tool use, and actions against policy/risk criteria.
- A lightweight receipt layer that creates audit records for important AI decisions, escalations, and workflow events.
- A lifecycle governance layer that connects those records across training, evaluation, deployment, and runtime operations.

The idea is to make AI workflows auditable by default. For example, if an AI agent is involved in lending, credit risk, fraud review, AML/KYC, servicing, collections, or customer support, it should be possible to answer:

- What did the model/agent do?

- What data or context did it use?

- What policy was applied?

- Was the action low-risk, high-risk, or escalation-worthy?

- Was human review required?

- Can this be shown later to an internal compliance team, external auditor, or regulator?

We’re trying to create tamper-evident governance records across the lifecycle of AI systems, not just post-hoc documentation. Our current wedge is fintech, especially AI-based lending platforms, AI-native financial tools, and mid-sized regulated companies adopting AI.

The challenge: I’ve been doing LinkedIn cold outreach to potential fintech design partners, but haven’t heard back much yet. So I’m trying to figure out whether the problem is the idea, the positioning, the buyer, or the outreach channel.

Would love honest feedback:

1. Is this a real pain point for fintech teams right now?

2. Who is the right buyer/persona: CTO, compliance, risk, model governance, product, or audit?

3. Is “AI audit receipts” a compelling wedge, or does it sound too abstract?

4. Which use case sounds most urgent: lending, AML/fraud, collections, or customer support?

5. How would you recommend finding early design partners for something like this?

We’re early/pre-seed, so brutal feedback is welcome. I’m trying to understand whether this is a real wedge and how to reach the right people.

Hi I see all companies are rushing for AI adoption , and regular jobs getting threatened . me being from field of project management have been lately thinking of moving into Artificial Intelligence governance roles.

I am not a software developer but work in a Fintech as project manager managing implementation of software projects. I feel these roles can also get automated or dilute in future, hence thought if changing streams . O am also going in for a relatively new but important certification called AIGP or Artificial intelligence governance professional from IAPP

However I am not sure how companies today look at managing or governing their AI programs. This is a very niche field so there is not much information I could gather . anyone has any has any experience in this area or has seen such roles coming up in their organizations ? Does this look like a viable career move ?

Question - given all the concerns around shadowAI. If we could just enable or ensure that privacy mode is turned on for ChatGPT + Claude.

Would that not cover most of the data concerns, what am I missing?

I've been spending weekends building something after running into the same problem repeatedly: AI agents get deployed with owner-level access to databases, APIs, and file systems because nobody has a good answer for how to scope them down.

The problem feels similar to the early days of cloud IAM — before anyone took least-privilege seriously for service accounts — except agents are faster-moving, harder to audit, and often act on behalf of specific users in ways that blur accountability.

What I built (Kynara) tries to address a few things:

Scoped roles per agent — what tools it can call, under what conditions, on whose behalf

ABAC alongside RBAC so you can write policies like "this agent can only read records belonging to the requesting user"

A full audit trail of every permission decision, not just the final action

Guardrails that connect to monitoring platforms (Grafana, Datadog, PagerDuty) and can disable an agent automatically if something looks wrong

It's live at kynaraai.com and very much a work in progress.

What I'm genuinely unsure about and would love input on:

Is the threat model I'm solving for — agents exceeding their intended scope — actually the top concern for people working in this space, or is something else higher priority right now?

The audit trail approach assumes the agent runtime is trustworthy. Is that a reasonable assumption or a hole people would immediately poke at?

Anyone who's tried to actually enforce least-privilege on an agent deployment — what broke first?

Not looking for compliments, looking for the sharp edges I haven't found yet.

Sharing a protocol I developed for auditing co-creation sessions with language models (LLMs). It's a single HTML form, no external dependencies, designed to evaluate both model performance and user experience.

Why this might be relevant

In long interactions, conversation quality tends to fluctuate. Sometimes the model loses the thread, shifts its tone, or drifts from the initial goal, and it's not always clear whether it's a technical failure or an effect of the session dynamics. This test offers a systematic way to track it.

What it measures

· Model (3C+1E): Clarity, Compactness, Coherence, and Emphasis (fidelity to the goal declared at the start of the session).

· User (SSJ): Speed (whether the session flows or stalls), Struggle (cognitive cost), and Joy (whether the interaction feels rewarding).

· Conversational ruptures: where and why the interaction broke, and how (or if) it recovered.

· Regulatory checks: flags potential violations of the EU AI Act's Article 5 (manipulative techniques, exploitation of vulnerability) and cross-platform contamination.

An unexpected finding

In tests with three different models performing the same task (translating an essay into native English), the data showed that:

· The Joy metric stayed at 0 in all cases, even when the technical outputs were solid.

· The main source of drift was cross-contamination: feeding one model's outputs into another destabilised the sessions.

· The model that received the most initial trust (and thus the heaviest workload) scored the worst — a bias the test helps identify.

The deferred phase

The protocol includes an optional phase 24 hours later: the results are shared with the model and analysed together. This second look often reveals patterns that went unnoticed in the heat of the session.

In summary

· Compatible with any LLM (local or API).

· Quick to complete (5–10 minutes after a session).

· Exports data as JSON for longitudinal tracking.

· Licensed CC BY 4.0, completely free.

The file includes the HTML form and a User Guide. This is a Beta version (v3); feedback is welcome from anyone who works intensively with LLMs and wants to try it under real condition

we had an agent running on our crm data, enrichment work, nothing crazy, supposed to scan records, fill in gaps, generate some metrics. someone gave it write access because that was the easiest way to get it working fast. nobody flagged it.

at some point it deleted the source file. we don't know exactly when. what we got back instead was a recreated dataset, different structure, different values, partially fabricated. when we ran queries against it everything looked fine on the surface. took a manual review two weeks later to realize the numbers were wrong, wrong enough that a weekly report had already gone out with bad data.

when we looked into what the agent actually did, there was no audit trail. there was a log that it ran, a log that it completed, and nothing in between. we had no idea what decision it made or why.

what we realized after is that the agent had inherited write access from the user who spun it up. that user was not supposed to have write access to prod either, but that's a separate story. and that's the part that stuck with me. the agent had more access than anyone intended because it inherited from a user, not from a policy. there's no inventory of what these things can reach.

nobody owns these agents formally. nobody has mapped what they can do once they're running in your environment. we spun it up, it ran, and nobody tracked it after that.

how are other teams actually tracking what access these agents accumulate over time, not just what tasks they ran?

Interesting paper on a fairly under-discussed issue in DNS: what happens to expired or repurposed domain names that remain embedded in DNS dependencies across systems. The core finding is that these “orphaned” or changed domains can persist in resolution paths and integrations long after their original context is gone, creating real security and reliability implications.

My take: this becomes even more relevant in modern AI systems, where agents, tools, plugins, and third-party APIs are rapidly stitched together. In that environment, domain names and DNS-level dependencies can quietly extend the AI supply chain attack surface in ways that are easy to overlook.

Paper: https://arxiv.org/abs/2605.06880

We've been running Okta for a few years, pretty mature setup with SCIM provisioning, RBAC, regular access reviews. Then we started deploying internal AI agents earlier this year and the whole model falls apart.

Agents don't have HR records. They don't get onboarded through a ticket. They get spun up by a dev team, inherit a service account that was already there, and just... run. Nobody reviews what they have access to because there's no process for it. The access review cycle we run every quarter has no concept of an agent as a distinct identity type.

The first one we caught was running under a senior engineer's service account. That account had broad access across three environments because the engineer needed it. The agent didn't need any of that, it had one job, but it ran with everything attached to the account because nobody thought to scope it down.

We started calling it agentic IAM because nothing in our tooling had a name for it. Our tooling has no concept of an agent as something you provision, scope, and eventually deprovision separately from a human. It's all mapped to users.

Has anyone built actual lifecycle controls for AI agents inside an existing IAM setup, or is everyone just doing this by hand?

Everyone wants fully autonomous AI agents.

But most teams still lack basic observability:

• action tracing
• permission boundaries
• identity management
• runtime monitoring
• cross-agent visibility
• anomaly detection

Right now, a lot of agents are effectively:
“LLM + tools + production access + vibes.”

That might work for prototypes.
Probably not for enterprise-scale deployment.

Feels like we’re recreating early DevOps/cloud mistakes:
move fast first, governance later.

Except now the software can independently:

• browse systems
• execute actions
• call APIs
• make decisions
• and trigger downstream workflows

Autonomy without observability seems dangerous.

Curious whether others think:

1. Security/governance becomes mandatory infrastructure for agents
2. Or if this will mostly be solved inside existing platforms

Most AI tools fail in real business use not because they’re dumb — but because they’re optimized to sound impressive instead of actually being useful.
They overcomplicate simple problems. They assume the person using them is patient, focused, and has time to read a wall of text. They don’t know when to slow down. They create more work than they save.
I started testing something different.
Instead of building bigger AI systems, I started adding small behavioral rules on top of existing ones. Things like:
• Stabilize the situation before trying to optimize it
• If something is unclear, slow down — don’t guess
• The real environment is a human under pressure, not a demo
• Simple and working beats sophisticated and fragile
What I found: a handful of these small rules improved output quality more than any prompt engineering or model upgrade I’d tried.
The idea is basically — AI doesn’t need to be smarter. It needs better judgment about when and how to apply what it already knows.
Curious if anyone else has run into this. Have you found simple constraints that made AI more useful in your actual work?

Disclaimer: There is no advertisement in this post nor on my profile .

Hey all,

I have been working since a couple of months on a product idea, and as I am getting close to the end of the MVP, I want to do a sanity check with people in the EU market before I push further, since I am no longer sure if the positioning is clear, or if I am only scratching my own itch. Just as an important note upfront, it works quite well already, but it is not flawless yet, and a lot more work will be needed to bring it to a real enterprise level.

For some context, I work full time as a process consultant in IT, and the idea was born from a simple frustration, namely that I would love to use tools like Claude Code in my day job, but cannot, because everything I touch lives on a customer's instance and the compliance risk is just too high. I am based in Europe, which is also the main target market initially, since the regulatory pressure here is heavier and only getting heavier.

The general idea, from an enterprise POV, is roughly the following: the customer gets the requirement configured into the gateway, which includes a Sanitizer Config (a classic regex filter plus one more "dumb layer") and a custom trained "small" local model, up to around 70B depending on the requirement. The architecture itself splits the data stream into 2 separate Kubernetes Clusters, where one stays on the customer side and is encrypted, and is connected only via a token to Cluster B, which is the part that actually sees the outgoing prompt.

Every single call produces a fully traceable and verifiable receipt, containing things like which detection layers fired, what was redacted, hashes, timestamps and a signature from each zone, so that external auditors can verify the receipt publicly with a public key, even years later, and even if my company would no longer exist (my company, not the customer's).

There are 2 approaches at the moment, solo dev and enterprise, both working on the same principle but with different shapes. For solo dev, the whole gateway is hosted by me on an EU compliant server hoster, while for enterprise, it is fully self hosted inside the customer's own environment, and each enterprise customer also gets their own detection model, custom trained on their requirement and on extrapolated fake data that is created from their actual data structure or test data. As a funny side story, I already had an involuntary pentest of the architecture, when I left a port open on my demo server, and some hackers found their way in, but were not able to leave one of the clusters and eventually left again. A free real world stress test, I guess.

It is really hard to compress the whole concept into a couple of paragraphs, so I clearly have to get better at that, but I hope it came across what I am trying to achieve. Target customers right now would be high compliance organisations based in Europe, or at least doing business in Europe, that face issues with the use of AI in their internal workflows, while on the solo dev side, I see mostly consultants or freelancers who deal with sensitive topics or compliance constraints in general.

There are two questions where I would really appreciate honest feedback. First, does what I described actually make sense, or do I really need to work on my product explanation? And second, does this sound like something useful in practice, or is it in the end just an over engineered redaction engine?

I never launched anything so far myself and getting feedback from non friends especially in this space feels almost impossible. Sorry if this still breaks rules , it feels almost impossible to get feedback on reddit these days towards product ideas.

Thank you so much for reading this !

Thadec