Do we need to allow each AI separately in robots.txt file, like ChatGPT and Perplexity?
Example:
User-agent: GPTBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Googlebot
Allow: /
User-agent: *
Disallow:
Example:
User-agent: GPTBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Googlebot
Allow: /
User-agent: *
Disallow:
Google just published its official AI Optimization Guide. Here's what every business owner actually needs to know.
After all the hand-wringing about AI Overviews and AI Mode killing SEO, Google's position is refreshingly clear: there are no special tricks, no secret schema, no AI text files needed.
The headline: AI features pull from the same index as classic Search. If your content earns its place in the top 10, it's eligible to be cited in AI Overviews and AI Mode.
Here's what Google actually says works:
✅ Allow crawling in robots.txt
✅ Build strong internal linking
✅ Deliver a great page experience
✅ Keep important content in text (not buried in images)
✅ Match your structured data to your visible content
✅ Keep your Google Business Profile current
✅ Create helpful, reliable, people-first content (E-E-A-T)
What you DON'T need:
❌ New machine-readable files
❌ Special AI markup
❌ A secret schema.org type
❌ A separate GEO playbook divorced from SEO
The interesting wrinkle? Google confirmed clicks coming from AI Overviews are higher quality. Users spend more time on those sites. That tracks with what I've been seeing across client accounts since AI Overviews rolled out.
My take, after writing SEO copy since 2008: this guide validates what I've been preaching. GEO, AEO, AIO, and AIEO are not separate disciplines. They are evolutions of solid SEO fundamentals. Helpful content, written by humans with real expertise, structured cleanly, served fast.
SEO didn't die. It had kids. And Google just told us they want to be raised the same way.
What's working for you in AI search right now?
I’ve been testing different AI-generated headline styles recently and noticed that most tools still default to:
They’re technically SEO-friendly, but they also feel predictable now.
I’ve been experimenting with generating titles using:
Curious from people actually working in SEO:
What type of titles are genuinely improving CTR for you right now?
Are classic SEO title structures still working, or are more curiosity-driven titles outperforming them?