We built an AI visibility tracker. Here's the 3 decisions that nearly broke us — and why we made them.
Building in public a bit here. We track how often brands
show up in ChatGPT, Perplexity, and Google AI answers.
Three decisions caused the most pushback from early users.
Curious if others have thought about this.
We only measure weekly. On purpose.
Early users wanted daily tracking. We said no.
Here's why: AI visibility doesn't move day-to-day.
What actually drives whether ChatGPT recommends you
is your content, citations, mentions, G2/Reddit reviews,
structured data. That stuff moves in weeks or months,
not overnight.
So if you measure Monday vs Tuesday, any difference
in your score is noise, not signal. The engine just
rolled the dice differently. Not because you improved.
Not because you got worse.
Daily tracking is expensive and misleading.
Weekly is a real signal.
Question: do you think weekly is enough?
Or would you want daily even knowing it's mostly noise?
We run each question 3 times and report a rate, not a yes/no.
AI engines are non-deterministic. Ask ChatGPT the exact
same question 3 times and you'll get 3 different answers,
often with different brands mentioned.
So "did you appear?" from a single scan is basically a coin flip.
We run each prompt 3 times and report: mentioned in 2 out of 3,
so 67% probability. That's a real estimate. A single answer is not.
Question: how do other AI tracking tools handle this?
Do they measure once and call it done?
Because that seems like a pretty broken metric to me.
We split prompts into layers and branded questions
don't count toward your main score.
Not all prompts are equal.
Buyer-intent, so "best issue tracker for dev teams",
is where you actually want to appear. Around 55% of scans.
Category questions like "how do small teams manage sprints"
cast a wider net, still unbranded.
Competitor-anchored prompts like "alternatives to Jira"
catch people who are actively shopping the competition.
And then branded prompts like "what is Linear?"
test whether AI actually knows you correctly.
The key decision: branded prompts are excluded
from your headline score.
Why? Because if the question already names you,
of course you'll appear. It would inflate your number
by 15 to 20 points artificially.
So branded questions get their own section:
does AI know you accurately? Not: how visible are you?
Question: does this split make sense to you?
Would you want branded included in the main score
or does that feel like gaming the metric?
Genuinely curious what people think. Especially if you've
tried to track AI visibility yourself and ran into
similar problems or solved them differently.
What's your take?