I have been trying to understand AI visibility tools a little better. Most of them appear to work with a set of questions, either written by the customer or generated from keywords, topics, brands, competitors, and related terms. That seems reasonable at first. But it made me wonder about a few things.

If a customer provides the keywords, themes, and brands that matter to them, how neutral can the resulting question set really be?

If the system generates questions from those inputs, is it discovering what people naturally ask, or is it creating questions inside the semantic space the customer already wants to be associated with?

And if the customer writes the questions directly, would they not usually formulate them in a way that makes the desired brand, product, or wording more likely to appear in the answer?

For example, there seems to be a big difference between asking:

>“How can I improve my email marketing?”

and asking:

>“How can Klaviyo improve my email marketing?”

Both may be valid questions. But in the second one, Klaviyo is already part of the framing before the AI has answered anything. The same applies more indirectly when questions are generated from a brand name, product category, target keywords, competitors, or desired associations.

So when a report says that a brand appeared in 18 out of 25 AI answers, what do those 25 questions represent?

Are they a realistic sample of what people would independently ask?

Are they based on actual user demand?

Are they generated from the customer's desired associations?

Or are they simply the questions most likely to make a specific brand or product relevant?

I am not saying that this makes AI visibility meaningless. It may still be useful to measure whether a model connects a brand with a topic, whether it recognises a product category, or whether it mentions a company once a user has already framed the problem in a certain way.

But should all of that be interpreted as the same thing?

At what point are we measuring independent AI visibility, and at what point are we measuring the effect of the questions we chose before the AI ever answered?

I am trying to understand where the line is between a useful AI visibility signal and actual proof.

I understand why prompt tracking is valuable. If you track a set of relevant prompts, compare brand mentions, citations and competitors over time, you can clearly see how a model responds under those conditions.

But isn't that still mainly a controlled simulation?

It proves that a model gave a certain answer to a prompt we chose, at a specific time. It does not necessarily prove that real users asked comparable questions, saw the answer, clicked a source, visited the site, or changed a buying decision because of it.

At the same time:

• a citation does not necessarily mean a click
• referral traffic does not necessarily prove influence from an AI answer
• a crawler or search bot visiting a page does not necessarily prove that the page appeared in a user-facing answer
• a conversion does not necessarily prove AI was the cause

So when a client asks: “Can you prove that our brand is actually being seen or used through AI?”, what is the honest answer?

Is the industry generally comfortable treating AI visibility as a directional proxy, similar to a structured market simulation? Or is there already a more rigorous way to separate:

1. what we tested,
2. what an AI system may have accessed,
3. what users may actually have seen,
4. and what had real business impact?

Not trying to attack AI visibility tools here. They can obviously be useful. I am genuinely wondering whether we are sometimes presenting a stronger level of certainty than the underlying methodology can support.

Your LiteSpeed cache crawler may be wasting server resources

If you use WordPress with LiteSpeed Cache, you may have enabled the crawler / cache warmup feature. The idea sounds good:

LiteSpeed visits your pages before real visitors do, so the cache is already warm. But here is the part many users probably never check:

Which URLs is LiteSpeed actually warming?

In many setups, the crawler uses the sitemap as its URL source. That sounds normal. But it can also be extremely wasteful. A sitemap is made for search engines. It lists URLs that exist or should be discoverable. But cache warmup has a different job.

Cache warmup should not ask:

Which URLs exist?

It should ask:

Which URLs do real visitors actually need?

Because every warmup request costs server resources.

Your server has to process PHP, WordPress, plugins, theme code, database queries, cache logic and everything else needed to generate the page. If the crawler warms 1,000 sitemap URLs, but real visitors only ever request 300 of them, then a big part of the warmup is just self-inflicted load. You are spending server resources on pages nobody asked for. That is not optimization.

That is waste. And it can be worse than that.

Some URLs that real users actually visit may not even be in the sitemap. Pagination, filtered pages, category navigation or other real browsing paths are often not covered properly. So the crawler may warm URLs nobody needs, while missing URLs people actually request. The problem is not LiteSpeed Cache itself. The problem is the URL source. A cache crawler is only as useful as the URLs you feed into it. That is why I think sitemap-based warmup is often the wrong default.

A better approach would be simple:

Let real visitor requests decide what should be warmed. Collect URLs that were actually requested by guests.

Ignore logged-in users, admin pages, AJAX requests, obvious bots, non-cacheable URLs and technical noise. Then generate a dedicated warmup sitemap from those real requests. LiteSpeed Cache already supports additional sitemap sources for its crawler, so this would not need to replace LiteSpeed Cache. It would just give the crawler a smarter list.

This is not just a theoretical idea.

A similar demand-based URL collection approach has been used for years with Kitt, but so far it has only been available inside that specific cache crawler ecosystem. Now I am considering whether it would be worth the effort to make this URL-discovery part available as a small standalone helper plugin for normal LiteSpeed Cache users who do not use alternative cache crawler.

The workflow would be simple:

Install the helper plugin.

• Let it collect real cacheable guest-requested URLs.
• Add its generated sitemap to LiteSpeed Cache as an additional crawler source.
• Let LiteSpeed warm pages people actually use.
• That is it.

No big dashboard. No magic performance promise. No replacement for LiteSpeed Cache. Just a better source for cache warmup.

Because honestly:

If cache warmup creates load, it should at least warm the pages that matter. Not every URL that happens to exist in a sitemap.

So my real question is:

Would this provide actual value for LiteSpeed Cache users?

Would you use a small install-and-forget plugin that creates a demand-based warmup sitemap from real visitor requests?

Or is sitemap-based warmup good enough for your site?

I am trying to figure out whether it is worth making this available outside of Kitt for all LiteSpeed Cache users, or whether this is one of those ideas that sounds useful in theory but is not something people would actually install.

Maybe I’m missing something obvious here, so sorry if this is a dumb question.

I keep seeing more tools and discussions around “AI visibility”, “GEO”, “brand visibility in ChatGPT/Perplexity/etc.” and similar things.

As far as I understand it, many of these tools check prompts and then report whether your brand, website, or URL was mentioned or cited in the AI answer.

But what exactly is the proof part?

For example, if a tool asks ChatGPT or Perplexity a question and my brand shows up, that proves the tool got that answer during that test.

But does it prove that real users see the same thing?

And if my website is cited somewhere, does that prove the AI system actually accessed my website, or only that the URL appeared in the answer?

I’m not trying to criticize the tools. I’m just trying to understand the difference between:

• testing prompts
• being mentioned in an AI answer
• being cited as a source
• proving that an AI system actually visited or used your website

Are these basically the same thing in practice, or are people using “evidence” in a softer sense here?

I keep thinking about what AI search means for normal WordPress website owners.

For years, the deal was more or less simple:

Google crawls your site, shows your content in search results, and if people are interested, they click through to your website.

That was not always perfect, but at least there was a visible path:

crawl → index → ranking → impression → click → visitor

With AI answers, this path feels much less clear.

- A crawler may visit your site.
- Your content may help generate an answer.
- Maybe your site is cited.
- Maybe it is only used in the background.
- Maybe the user gets what they need without ever visiting your website.

At that point, the website still exists, but its role changes. It is no longer necessarily the place where people consume your content. It may become a supplier for answers shown somewhere else.

The usual reaction seems to be:

“Then just block all AI bots.”

I understand that reaction. But I’m not sure it is a real long-term strategy.

Blocking everything may protect your content from some uses, but it may also cut you off from future AI visibility. Allowing everything blindly is not great either, because site owners often have no idea what happened after a bot requested a page.

So the real problem may not be “allow or block”.

The real problem is that website owners are forced to make that decision almost blind. What we seem to be missing is some kind of evidence layer between AI crawler access and actual AI value.

Not just:

“Did a bot request my URL?”

But:

- Was it training, search, retrieval, or user-triggered?
- Was my content cited or mentioned?
- Was it shown as a source?
- Did any traffic come back?
- Was there any measurable value for the publisher?

Without something like that, I don’t see how WordPress publishers are supposed to make a rational decision.

Blocking everything is a panic button. Allowing everything is blind trust.

There should be something in between. What is your opinion?

A discussion with friends and acquaintances made me think about AI bots and WordPress sites. They told me that Google, OpenAI, Anthropic and others run bots that crawl websites, partly for AI training, search, retrieval, or whatever else happens behind the scenes. They also said that allowing these bots might become important if you want your content to be mentioned, cited, or used in AI answers.

But from a normal WordPress site owner’s perspective, I see a basic problem:

How would I even know?

WordPress itself does not show me whether GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot or similar bots visited my site. Analytics usually shows human visitors and referral traffic, not raw crawler requests.

Many WordPress users also do not have easy access to raw server logs. And even if they do, they would still need to know which user agents belong to which AI crawlers and what those requests actually mean.

But the bigger question for me is this:

Even if I could see that an AI bot requested one of my URLs, how would I know whether my site was ever mentioned, cited, used as a source, or included in an AI answer?

With Google Search, we have Search Console.
With visitors, we have analytics.
With WordPress, we have plugins for almost everything.

But is there anything practical for AI crawler visibility or AI mentions?

Are WordPress site owners currently tracking this somehow, or are we basically blind unless we have server-log access and know how to interpret AI crawler requests?

>The view count suggests people are at least curious, but the lack of concrete answers suggests the tracking side is still unclear.

[Update]

That makes me wonder how SEO people are supposed to measure this reliably. With classic search there is at least some kind of chain: crawling, indexing, rankings, impressions, clicks. With AI answers, I don’t see the same chain yet. If crawler visits, mentions, citations and referrals are all separate signals, how do we know what actually worked?

In my previous post, I asked why WordPress sites should allow AI crawlers when they create load, take content and send little traffic back:
https://www.reddit.com/r/Wordpress/comments/1sxvtqz/wordpress_sites_why_allow_ai_crawlers_if_they/

AI crawlers feed chatbots. AI companies make money. Publishers get two bad choices:
block them, or hope.

Are you already blocking - or just hoping?

Do you know how much they scrape?

Google is the hardest case: the AI purpose is masked behind normal Googlebot activity.

OpenAI and Anthropic are different. Their bot descriptions help, but are too vague for real selective blocking. Blocking only by bot name may even remove useful signals.

Would you want to decide by bot, URL and request intent - instead of blindly blocking whole crawler families?

In my previous post, I asked why WordPress sites should allow AI crawlers when they create load, take content and send little traffic back:
https://www.reddit.com/r/Wordpress/comments/1sxvtqz/wordpress_sites_why_allow_ai_crawlers_if_they/

AI crawlers feed chatbots. AI companies make money. Publishers get two bad choices:
block them, or hope.

Are you already blocking - or just hoping?

Do you know how much they scrape?

Google is the hardest case: the AI purpose is masked behind normal Googlebot activity.

OpenAI and Anthropic are different. Their bot descriptions help, but are too vague for real selective blocking. Blocking only by bot name may even remove useful signals.

Would you want to decide by bot, URL and request intent - instead of blindly blocking whole crawler families?

Most WordPress traffic plugins give you a visitor view. But your server sees something else.

A lot of WordPress traffic is not clean human traffic. It can be crawlers, bots, scrapers, fake user agents, missing user agents, probe requests, AI crawlers, broken requests, suspicious paths, and other machine-driven noise.

Depending on the site and the measurement layer, this machine-driven request layer can become a surprisingly large part of what actually hits your server.

The problem is: many traffic plugins are built around the idea of counting visits. They are not built to help you inspect the suspicious layer.

That is why I built STV - Suspicious Traffic Viewer.

STV does not try to be another analytics dashboard. It does not replace server logs, a WAF, Cloudflare, or Wordfence. It also does not block anything.

Its purpose is simpler:

Make suspicious, bot-like, crawler-like, masked, and not-human-like WordPress traffic easier to see.

It focuses on things such as:

• missing or unusual user agents
• known bots and AI crawlers
• suspicious request paths
• non-200 responses
• requests that do not look like ordinary human pageviews
• traffic patterns that normal visitor analytics usually do not explain well

The plugin is currently available as a public GitHub release while the WordPress.org submission is still pending review.

GitHub:
https://github.com/LiteCache/litecache-stv

Important: download the installable plugin ZIP from the Releases section, not from GitHub’s “Code > Download ZIP” button.

A discussion with friends drew my attention to a specific topic: they told me that the majority of traffic is generated by bots or by disguised, machine-driven requests that cannot be detected by GA4, Matomo, or other traffic analysis tools. Is there a WordPress plugin that can make such "suspicious" traffic visible?

To clarify: by “suspicious” I don’t necessarily mean malicious or security-related traffic or bad bots.

I mean requests that may look technically normal, but do not behave like real human visitors. Wordfence can show and flag security-relevant traffic, but I’m interested in the layer between raw server logs and security alerts: requests that are invisible to GA4/Matomo and need behavioral context.

Following up on my earlier post about AI crawlers and masked traffic, I turned the idea into a free WordPress plugin.

https://www.reddit.com/r/Wordpress/comments/1sxvtqz/wordpress_sites_why_allow_ai_crawlers_if_they/

LiteCache Suspicious Traffic Viewer (STV) is not a generic traffic viewer and not a realtime logger. It focuses on suspicious traffic that often looks normal, blends in, or only becomes questionable when observed over time.

This is also not some speculative reaction to the current AI crawler hype. The logic behind STV is based on about 10 years of practical validation. AI crawlers did not create the problem. They mainly made it harder to ignore.

Another reason behind STV is that conventional traffic analyzers like GA4 or Matomo may see a lot, but still not everything. Especially not the traffic that deliberately avoids the very mechanisms those analyzers rely on.

Masked requests are only part of the problem. A much bigger issue is simple direct HTTP request traffic that bypasses typical analytics tracking completely, while still generating a large number of requests and unnecessary server load. That is not theory for me. It is part of my daily work when 1 human-like request can be accompanied by 500 clearly not-human-like requests.

STV is db-less while logging, but it has a clear limitation: requests fully served by page cache or CDN cache remain invisible.

The plugin will be available in the WordPress Plugin Repository soon. Until then, I can share the readme.txt with anyone interested.

If you want to read it or give feedback before release, let me know.

STV Plugin readme.txt: https://www.litecache.dev/readme.txt

LiteCache Suspicious Traffic Viewer Makes The Invisible Visible!

First Preview:

https://preview.redd.it/dsxlmhgh150h1.png?width=1333&format=png&auto=webp&s=270dad46822818f27319f84a59fe16f97b105986