We're measuring API outputs and calling it user exposure
Those are two different distributions.
Something has been bothering me since publishing two AI-visibility studies, and two commenters here put their finger on it independently.
Every AEO monitoring setup I know of — mine included — samples a model under controlled conditions: clean session, no history, no memory, no personalisation, often via API. Call that distribution Ps(B | intent, model, standard context). What we then report is "brand X has 30% visibility."
But the distribution a real customer draws from is Pr(B | intent, model, user, real context) — logged-in account, query history, memory, geography, whatever the router does that day. The industry treats these as interchangeable. There's no empirical basis for that, and some evidence against it: ACL 2026 work on personalisation finds error classes that only appear with real users, and FSPO (ICLR 2026) reports a substantial drop transferring from synthetic benchmarks to human evaluation.
Two things worth separating, though, because I think this is where the argument usually collapses:
At the level of one user, Ps ≠ Pr, obviously. That's not interesting. The interesting claim is the aggregate one: does contextual noise cancel across thousands of users on the same commercial intent, so that the synthetic distribution approximates the real one? That's testable and, as far as I can tell, untested.
And my hunch is the answer isn't uniform — it should depend on market structure. In concentrated verticals where models converge on a few obvious leaders, the gap should be small. In fragmented ones it should be large: in my own jewelry data, ten models asked the same question shared an average of 0.91 brands out of five, and two thirds of brands were named by exactly one model. Where models already disagree that much with each other, personalisation has enormous room to move the answer, and a synthetic benchmark could be off by a lot.
Which raises the uncomfortable question: does an AI visibility score need a confidence interval attached to it, per vertical? A number without one is being read as market share.
How would you test it? The design I keep coming back to is real users running identical commercial prompts in their own accounts, versus the same prompts in clean sessions, then measuring the distance — share error, rank correlation, top-3 overlap. Expensive, and I haven't run it. Curious whether anyone has data on how far personalisation actually moves brand recommendations, or whether it mostly moves phrasing.