u/keyworddotcom

We’ve been noticing something interesting while tracking AI referral traffic.

ChatGPT (and similar LLMs) are sending users to URLs that don’t actually exist on the site, but are close enough to real pages that they look valid.

When users click those, they land on a 404 and drop off, and we end up losing high-intent traffic without even realizing it.

What’s happening here is that LLMs don’t always link to exact URLs. They sometimes generate paths that approximate the right page, but miss the exact structure.

If AI is sending you traffic, there’s a good chance some links are breaking like this

Quick way to check:

• Pull landing pages in GA4 filtered by ChatGPT / AI sources
• Export the URLs
• Run a crawl using the site audit tool 
• Check status codes
• Filter for 404s

Those are essentially hallucinated URLs.

Fix:

• Map those URLs to the closest real pages
• Implement 301 redirects

Feels like one of those early AI-search quirks that’s easy to miss if you’re not looking for it

The first place we’d start is Google Search Console. Filter for longer queries, around 30–35+ characters. What you’ll usually find is a bunch of messy, natural-language searches, which is useful because that’s pretty close to how people ask questions in ChatGPT, Perplexity, Gemini, etc.

For example, queries like “how to check my website ranking on Google,” “how to find what keywords competitors are using,” or “best way to track local rankings for clients” are basically raw prompts. You can clean them up slightly so they sound more like actual AI search prompts. So something messy like “best ai rank tracker agency team 5 people” becomes “What’s the best AI rank tracker for a 5-person SEO agency?”

From there, you can expand each prompt with ChatGPT or Perplexity. Take one seed question like “What’s the best AI rank tracker?” and look at the follow-up questions the tools suggest. You’ll usually find ideas around comparisons, features, pricing, and specific use cases. One prompt can quickly turn into questions like “Which AI rank tracker is most accurate?”, “What tools track AI Overviews?”, “What’s the best AI rank tracker for agencies?”, or “How do I track brand mentions in AI search?”

The last step is to pull language from real communities. Search your topic in Reddit, Slack groups, forums, LinkedIn comments, or wherever your audience talks. Look for question-style posts, tool comparisons, complaints, feature requests, and “is there a tool that…” comments. The comments are often more useful than the original post because that’s where people describe the actual problem in their own words.

For example, if someone says, “We need something that tracks both AI Overviews and ChatGPT citations,” that can become “What tools track both AI Overviews and ChatGPT citations?” That’s a much better prompt than something generic like “AI SEO tool.”

By the end, you’ll have a prompt cluster based on your own GSC data, LLM follow-up questions, real community language, and actual pain points. You can easily get 100+ solid prompts this way in under an hour without using a dedicated AI SEO tool.

Obviously, tools help once you want to scale tracking and reporting. But for the research phase, this manual method is a pretty good starting point.