u/Latter_Community_946

Deployed Agent 365 last week specifically for the shadow AI detection piece. Got the Intune prerequisites sorted, enrolled the fleet, flipped the detection policy on. Took about a day.

It found OpenClaw. One agent. That's it.

Meanwhile our devs are running Claude Desktop, Cody, Continue, Cursor, and a local Ollama instance on a staging box. None of it flagged. The detection page is telling us we're fine when we are very obviously not fine.

I get that it's a preview and Microsoft says coverage will expand. But right now the gap between what Agent 365 sees and what's actually running is hard to ignore.

Anyone else rolled this out and found the same thing?

Shipped a customer-facing agent a few months back. Had the standard safety guardrails in place, felt pretty good about it.

First week in prod, a customer asks "should I go with you or [competitor]" and our agent gives them a thoughtful comparison that ends with honestly for your use case they might be a better fit. Like WTF??

Generic moderation didnt flag it. It just violated a brand policy nobody had encoded anywhere. Turns out out-of-the-box guardrails protect against everyone else's problems, not yours.

Lesson learned here is if the policy lives in your head and not in your guardrails, it doesnt exist. Posting this in case it saves someone the awkward postmortem we had to sit through.

Started with 50 GenAI tools to block via proxy. Two weeks later found teams using 30+ new ones I'd never heard of. Traditional URL blocking is whack-a-mole at this point.

Anyone moved to semantic DLP that catches data leaks regardless of which AI tool? Looking at browser-native solutions vs trying to patch our SWG. Need something that scales without constant list updates.

What's working for you folks?