What should count as proof when an AI agent says it found a vulnerability?
I’ve been experimenting with supervised AI-assisted pentesting in authorised lab environments, and one question keeps coming up:
At what point should we actually accept an AI-generated finding as valid?
LLMs are very good at producing something that sounds like a vulnerability report.
That is obviously not the same thing as proving the vulnerability exists.
For a human pentester, I’d normally want enough evidence that somebody else can independently reproduce the issue.
For an agent, I think the bar should be at least as high.
For example, depending on the finding:
- exact request and response
- reproduction steps
- affected endpoint/parameter
- observed versus expected behaviour
- evidence showing impact
- clean verification request
- screenshots where useful
- relevant tool output
- enough context for another tester to reproduce it
I’m increasingly sceptical of AI pentesting benchmarks that simply count “vulnerabilities found”.
If the model says:
«This endpoint appears vulnerable to IDOR»
that should be worth zero until it actually demonstrates unauthorised access to another object and preserves the evidence.
Same for injection.
Generating a payload isn’t finding SQL injection.
Getting an error isn’t necessarily finding SQL injection.
You need a reproducible behavioural difference that supports the hypothesis.
I also think the agent shouldn’t be the sole judge of whether its own work constitutes proof.
Ideally there is some deterministic or independently reviewable layer between:
hypothesis → test → evidence → validated finding
The other interesting question is false negatives.
An agent that reports 15 genuine vulnerabilities and misses five is arguably much more useful than one that reports 20 but five of them collapse under manual verification.
For people who actually pentest:
What minimum evidence would you require before accepting a vulnerability found by an AI agent?
Would a raw request/response pair be enough?
Would you require an independent verification step?
And should the standard be different for something like reflected XSS versus an access-control or business-logic flaw?