Do we need to allow each AI separately in robots.txt file, like ChatGPT and Perplexity?
Is it’s mandatory
Example:
User-agent: GPTBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Googlebot
Allow: /
User-agent: *
Disallow:
Is it’s mandatory
Example:
User-agent: GPTBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Googlebot
Allow: /
User-agent: *
Disallow:
I’ve been testing exact buyer-intent queries about our SaaS category. Some competitors are stubbornly locked into every response, while our product only surfaces if I use incredibly specific, narrow phrasings.
I’m trying to know why the visibility logic is volatile, and how are you tracking these citation shifts systematically without manual prompting?
Update: Circling back in case anyone else needs a tool like this, GentrackAI is what I've found to be the most useful.
This is gaining more and more concensus among SEOs - that SEO (and PPC) doesnt need multi-channel marketing. This is more a demand-gen campaign to make people think they "need" social mentions for Google and AI.
If you're only getting to "ranking' on Google for citations now - chances are you've missed the boat. Only one page from each search index from Reddit and Youtube can really rank. If you want it to be your post - then standard SEO applies.
Parasitic SEO is not free SEO rankings because LinkedIn as high DA - most topics are already covered: LinkedIn gets over 1 billion clicks from Google.
Had a client call today where SEO reporting started feeling like straight-up gaslighting. Visibility is up, impressions increased, rankings improved… cool. Why is traffic moving like a dead cryptocurrency then?
I’m seeing more pages technically perform better in Search Console while getting worse actual engagement from search. Feels like Google now wants publishers to research, write, structure, and optimize content just so it can answer the query itself and send us an impression as payment. Appreciate the exposure king.
And honestly, impressions are becoming one of the funniest metrics in SEO now. “Your site appeared 480,000 times.” Amazing. That and $4 will get me coffee.
User journeys also feel completely broken now. People read the AI Overview, open Reddit, ask ChatGPT, refine the search, then maybe visit your site later if Mercury is in retrograde. Meanwhile agencies are still reporting rankings like it’s 2019.
Anybody else seeing this weird disconnect between visibility and actual traffic?