I wrote a tool to catch AI agents disabling RLS. Then I red teamed it and it failed badly.
Something I kept running into with AI-assisted code: you ask the agent to fix a query, it can't work out the policy, so it just disables RLS. Or a paywall check gets set to false so a demo works. Or email confirmation goes off during testing and never goes back on. The code looks fine afterwards. Nothing errors.
So I wrote a CLI that fails the build on that specific kind of change. MIT, no dependencies:
npx prodguard check --demo
npx prodguard check
The demo runs everything against a fake broken app, so you can see the output without pointing it at anything real.
12 checks. The ones that matter here:
- ALTER TABLE ... DISABLE ROW LEVEL SECURITY, or a table that never had it turned on
- service_role key in client code, or sitting behind VITE_ or NEXT_PUBLIC_
- email_confirm: true / auto-confirm in real auth paths
- Stripe webhook handler with no signature check
- JWT decoded but never verified
- Firebase rules left on allow read, write: if true, including the console's 30 day test mode
- live keys committed, recovery code files committed, DELETE FROM with no WHERE
Before publishing I spent a day trying to break it, mostly by throwing agents at it to red team the thing. It went badly.
The big one: it was printing the secrets it found. I'd written a redact() function and only wired it into one of the twelve rules. The other ten printed the whole matching line. So when it found a service_role JWT, it printed the JWT. To your terminal and into your CI logs. Bit of a problem for a tool whose whole pitch is protecting that key.
A 512KB file could lock CI up for 159 seconds from regex backtracking. The same broken pattern meant TRUNCATE TABLE never matched anything in the first place.
If you typo'd the path it printed "Nothing dangerous found" and exited 0. So prodguard check ./scr passes. For a build gate that's about the worst failure available.
24 false positives. The worst hit every Supabase project going: run supabase init in an empty folder, scan it, two HIGH findings straight off the stock config. One of those was my email rule firing on enable_confirmations under [auth.sms], which is the phone setting.
Then a red team pass wrote 29 deliberately vulnerable files and ran it. Zero findings, exit 0. Every rule was matching literal strings, so const [locked, setLocked] = useState(false) went straight through, and that's how an agent actually writes it.
All fixed in 0.5.0 with tests for each. Redaction happens in the rule runner now, so no individual rule can skip it.
https://github.com/Felix0731/prodguard
There's a walkthrough on the site too, if you've not run something like this before: https://prodguard.vercel.app
What it doesn't do: it reads text, it doesn't parse your code. Write the same bug a different way and it'll miss it. It gets things wrong sometimes too, there's an ignore list for that. A pass means those 12 checks didn't fire, nothing more. And anything you set in the dashboard instead of your repo it can't see, same as any repo scanner.
If an agent has broken something in your project that this doesn't catch, tell me what the diff looked like and I'll write a rule for it. That's genuinely the most useful thing I could get right now.