u/Good_Flight6250

When a website has performance problems, the usual response is to optimize what is already there.

Minify CSS.
Delay JavaScript.
Remove unused styles.
Compress assets.
Improve the PageSpeed score.

But there is a more basic question:

Why deliver something that a device does not need in the first place?

A phone does not necessarily need the same frontend resources as a desktop. Yet conventional responsive web design often sends the same CSS and JavaScript to every device and lets the browser decide which parts are actually used.

That works. But it also means we may spend time optimizing resources that never needed to be delivered.

There is an older concept called RESS - Responsive Design + Server-Side Components.

The idea is simple:

Desktop → desktop.css
Tablet → tablet.css
Phone → mobile.css

The server decides which resource set is appropriate before the page reaches the browser.

And this does not mean abandoning responsive web design. Each device-specific stylesheet can still use normal media queries and responsive techniques.

RESS simply adds an earlier decision:

First decide what the device actually needs. Then optimize what remains.

Maybe "performance optimized" should therefore mean more than minification, caching and delayed loading.

Maybe users should also start asking theme and plugin developers:

Does your solution avoid delivering frontend resources that the device will never need?

EDIT: Judging from some comments, my CSS example may have been too narrow. RESS is not about the server replacing media queries. I made a small demo showing the actual idea: the server delivers different PHP, HTML and assets depending on the device, while responsive design can still be used normally inside each version.

Demo: https://www.imedes.dev/ress/

Full disclosure: I’m the developer.

Most cache-crawler setups start with a normal sitemap. But an SEO sitemap answers “which URLs exist?”, not “which URLs are actually worth warming?”

URLs Most Wanted records the URLs real visitors request, tracks their demand, validates them server-side, and generates a separate sitemap for the built-in LiteSpeed Cache crawler.

The idea is simple:

- URLs with demonstrated visitor demand can be warmed first.

- Rarely used URLs do not have to consume crawler resources repeatedly.

- Cacheable visitor paths such as pagination or filtered listings can be included even when they do not belong in a normal SEO sitemap.

The plugin does not replace LiteSpeed Cache or its crawler. It only gives the crawler a demand-based source of URLs.

It is free and available on WordPress.org:

https://wordpress.org/plugins/urls-most-wanted-for-litespeed-lscache/

I’m curious whether demand-based cache warmup would be useful on the WordPress sites you manage, or whether you generally prefer to crawl every available URL.

https://preview.redd.it/212zspfv8kch1.png?width=1319&format=png&auto=webp&s=7c6f2ab21d3844397ce16d3bfebbf6e287a1b697

I’ve been looking again at something that has bothered me about Wordfence for years.

Wordfence runs long-running PHP processes, especially its scanner. On LiteSpeed servers, those processes must not simply be terminated when the client connection disappears or certain connection timeouts are reached.

That part is perfectly reasonable. The problem is how Wordfence handles it. Wordfence recommends adding a rule like this to .htaccess:

RewriteRule .* - [E=noabort:1]

or even:

SetEnv noabort 1

The important part here is .*.

This does not protect only the Wordfence scanner. It applies noabort to every matching PHP request on the site.

LiteSpeed itself explicitly warns against applying noabort globally and recommends restricting it to the specific scripts or requests that actually require long-running execution.

And there is a very good reason for that.

noabort changes how LiteSpeed handles PHP processes when a connection disappears. Together with noconntimeout, a PHP process can continue running far beyond what many administrators would normally expect.

And before someone points at PHP's max_execution_time: on LiteSpeed/LSPHP that is not necessarily the hard process lifetime limit people assume it is.

I have tested this myself. (I am a LiteSpeed developer)

A request can reach its configured PHP execution time while the associated process continues running. The LiteSpeed-side mechanism that can impose a hard process-time limit is LSAPI_MAX_PROCESS_TIME.

You can observe the difference directly at process level. So the issue is not that Wordfence needs noabort. The issue is this:

Why does a security plugin remove a server-side process protection globally when only a small number of its own requests actually need that exception?

And there is another part of this story that I think matters even more:

This is not a newly discovered edge case. Wordfence has been aware of this issue for at least a decade.

I raised this problem directly with Wordfence years ago. LiteSpeed has also made clear for a long time that global noabort should be avoided when the exception can be restricted to the requests that actually require it.

Yet the broad configuration is still being recommended. That history is what makes this particularly difficult to understand. Software contains mistakes. Security software contains mistakes too. That is not the issue.

The issue is when a security vendor is made aware that its own configuration unnecessarily weakens a server-side protection, the server vendor explicitly warns against that same configuration, a technically straightforward way to scope the exception exists, and the unsafe recommendation remains in place for years.

At that point, this is no longer just an overlooked configuration detail. It is a consciously unaddressed security trade-off imposed on Wordfence users. And technically, the solution is not complicated.

With mod_rewrite, noabort can be enabled only for the exact Wordfence request that actually requires it.

Conceptually:

RewriteCond %{QUERY_STRING} ...
RewriteRule ^wp-admin/admin-ajax\.php$ - [E=noabort:1,E=noconntimeout:1]

instead of:

RewriteRule .* - [E=noabort:1]

Same Wordfence functionality. Very different security boundary.

Millions of WordPress users install Wordfence specifically because they trust it to improve the security of their sites. A security plugin should not unnecessarily weaken process controls for unrelated PHP code just because one of its own components needs an exception. And if a security vendor has known about that problem for at least a decade and still does not fix it, users should at least have the option to fix it themselves.

I’ve been looking again at something that has bothered me about Wordfence for years.

Wordfence runs long-running PHP processes, especially its scanner. On LiteSpeed servers, those processes must not simply be terminated when the client connection disappears or certain connection timeouts are reached.

That part is perfectly reasonable. The problem is how Wordfence handles it. Wordfence recommends adding a rule like this to .htaccess:

RewriteRule .* - [E=noabort:1]

or even:

SetEnv noabort 1

The important part here is .*.

This does not protect only the Wordfence scanner. It applies noabort to every matching PHP request on the site.

LiteSpeed itself explicitly warns against applying noabort globally and recommends restricting it to the specific scripts or requests that actually require long-running execution.

And there is a very good reason for that.

noabort changes how LiteSpeed handles PHP processes when a connection disappears. Together with noconntimeout, a PHP process can continue running far beyond what many administrators would normally expect.

And before someone points at PHP's max_execution_time: on LiteSpeed/LSPHP that is not necessarily the hard process lifetime limit people assume it is.

I have tested this myself. (I am a LiteSpeed developer)

A request can reach its configured PHP execution time while the associated process continues running. The LiteSpeed-side mechanism that can impose a hard process-time limit is LSAPI_MAX_PROCESS_TIME.

You can observe the difference directly at process level. So the issue is not that Wordfence needs noabort. The issue is this:

Why does a security plugin remove a server-side process protection globally when only a small number of its own requests actually need that exception?

And there is another part of this story that I think matters even more:

This is not a newly discovered edge case. Wordfence has been aware of this issue for at least a decade.

I raised this problem directly with Wordfence years ago. LiteSpeed has also made clear for a long time that global noabort should be avoided when the exception can be restricted to the requests that actually require it.

Yet the broad configuration is still being recommended. That history is what makes this particularly difficult to understand. Software contains mistakes. Security software contains mistakes too. That is not the issue.

The issue is when a security vendor is made aware that its own configuration unnecessarily weakens a server-side protection, the server vendor explicitly warns against that same configuration, a technically straightforward way to scope the exception exists, and the unsafe recommendation remains in place for years.

At that point, this is no longer just an overlooked configuration detail. It is a consciously unaddressed security trade-off imposed on Wordfence users. And technically, the solution is not complicated.

With mod_rewrite, noabort can be enabled only for the exact Wordfence request that actually requires it.

Conceptually:

RewriteCond %{QUERY_STRING} ...
RewriteRule ^wp-admin/admin-ajax\.php$ - [E=noabort:1,E=noconntimeout:1]

instead of:

RewriteRule .* - [E=noabort:1]

Same Wordfence functionality. Very different security boundary.

Millions of WordPress users install Wordfence specifically because they trust it to improve the security of their sites. A security plugin should not unnecessarily weaken process controls for unrelated PHP code just because one of its own components needs an exception. And if a security vendor has known about that problem for at least a decade and still does not fix it, users should at least have the option to fix it themselves.

I've noticed that my server receives a surprisingly large number of requests every day for random URLs that don't exist.

Every one of those requests eventually results in a 404, but WordPress still has to bootstrap, load plugins and process the request before it can determine that the page doesn't exist.

With enough of these requests, that seems like a lot of unnecessary PHP work for something that will only return a 404 anyway.

How do you deal with this?

Do you just accept it as normal WordPress overhead, handle these requests before they reach WordPress, or use some other approach?

https://preview.redd.it/xim0mbdcovkh1.png?width=1491&format=png&auto=webp&s=9af8d54db2c73f9fb3095a8fec8feee11ede0e8f

I've noticed that my server receives a surprisingly large number of requests every day for random URLs that don't exist.

Every one of those requests eventually results in a 404, but WordPress still has to bootstrap, load plugins and process the request before it can determine that the page doesn't exist.

With enough of these requests, that seems like a lot of unnecessary PHP work for something that will only return a 404 anyway.

How do you deal with this?

Do you just accept it as normal WordPress overhead, handle these requests before they reach WordPress, or use some other approach?

https://preview.redd.it/4k06jdd5ovkh1.png?width=1491&format=png&auto=webp&s=03eb110d01cb42c41a8b9887fd9de3d5d472f514

SEOs have been taught for decades that URLs should be descriptive. A URL such as /wordpress-performance-optimization/ is considered better than something meaningless like /x7ab31/.

But that assumes the crawler actually needs to fetch the page before deciding what it is about.

I've been watching AI crawler behavior more closely, particularly GPTBot and ClaudeBot, and something made me question that assumption.

If a crawler can infer enough about the likely content from the URL, link context and surrounding semantics, it can also decide that the page isn't worth fetching - without ever seeing the actual content.

So I tested the opposite approach.

I exposed alternative Markdown representations of existing content through completely opaque URLs. The URLs contained no topic, keyword or other clue about what was behind them.

Both GPTBot and ClaudeBot discovered and fetched all of them.

This obviously doesn't prove that the content is used for training, citations or AI answers. It only proves something much earlier in the chain: the content was actually retrieved.

And that made me wonder whether GEO is currently starting too late.

Most GEO advice focuses on optimizing content for AI systems after discovery: structure, entities, concise answers, citations, schema, etc.

But what if the first optimization layer should be:

Make sure the AI crawler actually sees the content in the first place.

In that context, a descriptive URL may not always be an advantage. It may also give a crawler enough information to reject the content before fetching it.

Maybe AI discovery needs to be treated as its own optimization layer, separate from traditional SEO.

Has anyone else observed crawler selection happening before the actual page fetch?

I have been thinking about a methodological problem with AI visibility testing. I recently asked ChatGPT a very simple question about whether it knew a relatively niche product.

On one occasion, the answer was simply no.

When I repeated almost the same question later, ChatGPT performed a web search and returned a detailed, mostly accurate explanation with sources. The difference was not merely wording or normal response variation. The two answers represented opposite informational states:

• without search, the product was apparently unknown
• with search, the product was correctly identified and explained

This raises a question about how AI visibility platforms conduct their tests.

Using an API, it is possible to define the model, tools, reasoning settings and whether web search is available. That creates a more controlled test environment.

But does that environment actually represent what normal users experience in ChatGPT?

The consumer interface may decide internally whether to use model knowledge, web search, cached or indexed information, or another response route. The tester may not know or control those conditions.

There are also indications that consumer AI products can change modes depending on availability. Gemini, for example, recently informed me that its Pro mode was heavily used and that my request would be handled by a simpler mode instead.

That does not prove that server load caused my ChatGPT results. But it does show that the system processing a prompt may not always remain identical, even when the visible product and prompt appear unchanged.

This becomes especially problematic in comparison prompts.

Imagine that five brands are already present in the model's internal knowledge, while another five enter the answer only through live search. They may appear in the same comparison, but they are not being evaluated from the same informational starting point.

The established brands may benefit from learned reputation and existing associations. The searched brands depend on whether search was triggered, which sources were retrieved and how those sources framed them.

So what does an AI visibility score actually represent under these conditions?

A controlled API benchmark can measure how brands perform under a specific API configuration. A consumer-interface test can observe what a user happened to receive under largely unknown internal conditions.

Both observations may be useful. But they are not necessarily measuring the same thing.

How do current AI visibility tools account for this difference?

Do they separate model recall from search-based retrieval?

Do they disclose whether results were produced through the API or the consumer interface?

And can an API configuration realistically represent the variable conditions under which ordinary ChatGPT users receive answers?

Full disclosure: I’m the developer.

Most cache-crawler setups start with a normal sitemap. But an SEO sitemap answers “which URLs exist?”, not “which URLs are actually worth warming?”

URLs Most Wanted records the URLs real visitors request, tracks their demand, validates them server-side, and generates a separate sitemap for the built-in LiteSpeed Cache crawler.

The idea is simple:

- URLs with demonstrated visitor demand can be warmed first.

- Rarely used URLs do not have to consume crawler resources repeatedly.

- Cacheable visitor paths such as pagination or filtered listings can be included even when they do not belong in a normal SEO sitemap.

The plugin does not replace LiteSpeed Cache or its crawler. It only gives the crawler a demand-based source of URLs.

It is free and available on WordPress.org:

https://wordpress.org/plugins/urls-most-wanted-for-litespeed-lscache/

I’m curious whether demand-based cache warmup would be useful on the WordPress sites you manage, or whether you generally prefer to crawl every available URL.

https://preview.redd.it/212zspfv8kch1.png?width=1319&format=png&auto=webp&s=7c6f2ab21d3844397ce16d3bfebbf6e287a1b697

Full disclosure: I’m the developer.

Most cache-crawler setups start with a normal sitemap. But an SEO sitemap answers “which URLs exist?”, not “which URLs are actually worth warming?”

URLs Most Wanted records the URLs real visitors request, tracks their demand, validates them server-side, and generates a separate sitemap for the built-in LiteSpeed Cache crawler.

The idea is simple:

- URLs with demonstrated visitor demand can be warmed first.

- Rarely used URLs do not have to consume crawler resources repeatedly.

- Cacheable visitor paths such as pagination or filtered listings can be included even when they do not belong in a normal SEO sitemap.

The plugin does not replace LiteSpeed Cache or its crawler. It only gives the crawler a demand-based source of URLs.

It is free and available on WordPress.org:

https://wordpress.org/plugins/urls-most-wanted-for-litespeed-lscache/

I’m curious whether demand-based cache warmup would be useful on the WordPress sites you manage, or whether you generally prefer to crawl every available URL.

https://preview.redd.it/212zspfv8kch1.png?width=1319&format=png&auto=webp&s=7c6f2ab21d3844397ce16d3bfebbf6e287a1b697

I released a free WordPress plugin for a fairly specific LiteSpeed problem.

The LiteSpeed Cache crawler can warm the URLs provided by a sitemap, but a normal SEO sitemap has no knowledge of actual visitor demand. It may contain many URLs that are rarely requested, while excluding cacheable visitor paths such as pagination or filtered listings.

URLs Most Wanted records visitor-requested URLs asynchronously, tracks their hit counts, validates them against the LiteSpeed cache environment, and generates a separate sitemap for the LSCache crawler.

It does not modify or replace the crawler. It only changes the URL source:

normal sitemap = URLs that exist

UMW sitemap = URLs with demonstrated demand

The goal is to reduce unnecessary crawler work and focus warmup resources on URLs that visitors actually use, especially on hosting where crawler load matters.

Requirements are intentionally narrow: LiteSpeed Enterprise or OpenLiteSpeed at the origin, together with LiteSpeed Cache for WordPress.

Plugin page: https://wordpress.org/plugins/urls-most-wanted-for-litespeed-lscache/

Would you use a demand-based crawler sitemap, or do you prefer warming every discoverable URL?

https://preview.redd.it/fvl42v739kch1.png?width=1319&format=png&auto=webp&s=c3950c6f63aa7697676ad65ef81f4a8844d1215b

I have been trying to understand AI visibility tools a little better. Most of them appear to work with a set of questions, either written by the customer or generated from keywords, topics, brands, competitors, and related terms. That seems reasonable at first. But it made me wonder about a few things.

If a customer provides the keywords, themes, and brands that matter to them, how neutral can the resulting question set really be?

If the system generates questions from those inputs, is it discovering what people naturally ask, or is it creating questions inside the semantic space the customer already wants to be associated with?

And if the customer writes the questions directly, would they not usually formulate them in a way that makes the desired brand, product, or wording more likely to appear in the answer?

For example, there seems to be a big difference between asking:

>“How can I improve my email marketing?”

and asking:

>“How can Klaviyo improve my email marketing?”

Both may be valid questions. But in the second one, Klaviyo is already part of the framing before the AI has answered anything. The same applies more indirectly when questions are generated from a brand name, product category, target keywords, competitors, or desired associations.

So when a report says that a brand appeared in 18 out of 25 AI answers, what do those 25 questions represent?

Are they a realistic sample of what people would independently ask?

Are they based on actual user demand?

Are they generated from the customer's desired associations?

Or are they simply the questions most likely to make a specific brand or product relevant?

I am not saying that this makes AI visibility meaningless. It may still be useful to measure whether a model connects a brand with a topic, whether it recognises a product category, or whether it mentions a company once a user has already framed the problem in a certain way.

But should all of that be interpreted as the same thing?

At what point are we measuring independent AI visibility, and at what point are we measuring the effect of the questions we chose before the AI ever answered?

I am trying to understand where the line is between a useful AI visibility signal and actual proof.

I understand why prompt tracking is valuable. If you track a set of relevant prompts, compare brand mentions, citations and competitors over time, you can clearly see how a model responds under those conditions.

But isn't that still mainly a controlled simulation?

It proves that a model gave a certain answer to a prompt we chose, at a specific time. It does not necessarily prove that real users asked comparable questions, saw the answer, clicked a source, visited the site, or changed a buying decision because of it.

At the same time:

• a citation does not necessarily mean a click
• referral traffic does not necessarily prove influence from an AI answer
• a crawler or search bot visiting a page does not necessarily prove that the page appeared in a user-facing answer
• a conversion does not necessarily prove AI was the cause

So when a client asks: “Can you prove that our brand is actually being seen or used through AI?”, what is the honest answer?

Is the industry generally comfortable treating AI visibility as a directional proxy, similar to a structured market simulation? Or is there already a more rigorous way to separate:

1. what we tested,
2. what an AI system may have accessed,
3. what users may actually have seen,
4. and what had real business impact?

Not trying to attack AI visibility tools here. They can obviously be useful. I am genuinely wondering whether we are sometimes presenting a stronger level of certainty than the underlying methodology can support.

Your LiteSpeed cache crawler may be wasting server resources

If you use WordPress with LiteSpeed Cache, you may have enabled the crawler / cache warmup feature. The idea sounds good:

LiteSpeed visits your pages before real visitors do, so the cache is already warm. But here is the part many users probably never check:

Which URLs is LiteSpeed actually warming?

In many setups, the crawler uses the sitemap as its URL source. That sounds normal. But it can also be extremely wasteful. A sitemap is made for search engines. It lists URLs that exist or should be discoverable. But cache warmup has a different job.

Cache warmup should not ask:

Which URLs exist?

It should ask:

Which URLs do real visitors actually need?

Because every warmup request costs server resources.

Your server has to process PHP, WordPress, plugins, theme code, database queries, cache logic and everything else needed to generate the page. If the crawler warms 1,000 sitemap URLs, but real visitors only ever request 300 of them, then a big part of the warmup is just self-inflicted load. You are spending server resources on pages nobody asked for. That is not optimization.

That is waste. And it can be worse than that.

Some URLs that real users actually visit may not even be in the sitemap. Pagination, filtered pages, category navigation or other real browsing paths are often not covered properly. So the crawler may warm URLs nobody needs, while missing URLs people actually request. The problem is not LiteSpeed Cache itself. The problem is the URL source. A cache crawler is only as useful as the URLs you feed into it. That is why I think sitemap-based warmup is often the wrong default.

A better approach would be simple:

Let real visitor requests decide what should be warmed. Collect URLs that were actually requested by guests.

Ignore logged-in users, admin pages, AJAX requests, obvious bots, non-cacheable URLs and technical noise. Then generate a dedicated warmup sitemap from those real requests. LiteSpeed Cache already supports additional sitemap sources for its crawler, so this would not need to replace LiteSpeed Cache. It would just give the crawler a smarter list.

This is not just a theoretical idea.

A similar demand-based URL collection approach has been used for years with Kitt, but so far it has only been available inside that specific cache crawler ecosystem. Now I am considering whether it would be worth the effort to make this URL-discovery part available as a small standalone helper plugin for normal LiteSpeed Cache users who do not use alternative cache crawler.

The workflow would be simple:

Install the helper plugin.

• Let it collect real cacheable guest-requested URLs.
• Add its generated sitemap to LiteSpeed Cache as an additional crawler source.
• Let LiteSpeed warm pages people actually use.
• That is it.

No big dashboard. No magic performance promise. No replacement for LiteSpeed Cache. Just a better source for cache warmup.

Because honestly:

If cache warmup creates load, it should at least warm the pages that matter. Not every URL that happens to exist in a sitemap.

So my real question is:

Would this provide actual value for LiteSpeed Cache users?

Would you use a small install-and-forget plugin that creates a demand-based warmup sitemap from real visitor requests?

Or is sitemap-based warmup good enough for your site?

I am trying to figure out whether it is worth making this available outside of Kitt for all LiteSpeed Cache users, or whether this is one of those ideas that sounds useful in theory but is not something people would actually install.

Maybe I’m missing something obvious here, so sorry if this is a dumb question.

I keep seeing more tools and discussions around “AI visibility”, “GEO”, “brand visibility in ChatGPT/Perplexity/etc.” and similar things.

As far as I understand it, many of these tools check prompts and then report whether your brand, website, or URL was mentioned or cited in the AI answer.

But what exactly is the proof part?

For example, if a tool asks ChatGPT or Perplexity a question and my brand shows up, that proves the tool got that answer during that test.

But does it prove that real users see the same thing?

And if my website is cited somewhere, does that prove the AI system actually accessed my website, or only that the URL appeared in the answer?

I’m not trying to criticize the tools. I’m just trying to understand the difference between:

• testing prompts
• being mentioned in an AI answer
• being cited as a source
• proving that an AI system actually visited or used your website

Are these basically the same thing in practice, or are people using “evidence” in a softer sense here?

I keep thinking about what AI search means for normal WordPress website owners.

For years, the deal was more or less simple:

Google crawls your site, shows your content in search results, and if people are interested, they click through to your website.

That was not always perfect, but at least there was a visible path:

crawl → index → ranking → impression → click → visitor

With AI answers, this path feels much less clear.

- A crawler may visit your site.
- Your content may help generate an answer.
- Maybe your site is cited.
- Maybe it is only used in the background.
- Maybe the user gets what they need without ever visiting your website.

At that point, the website still exists, but its role changes. It is no longer necessarily the place where people consume your content. It may become a supplier for answers shown somewhere else.

The usual reaction seems to be:

“Then just block all AI bots.”

I understand that reaction. But I’m not sure it is a real long-term strategy.

Blocking everything may protect your content from some uses, but it may also cut you off from future AI visibility. Allowing everything blindly is not great either, because site owners often have no idea what happened after a bot requested a page.

So the real problem may not be “allow or block”.

The real problem is that website owners are forced to make that decision almost blind. What we seem to be missing is some kind of evidence layer between AI crawler access and actual AI value.

Not just:

“Did a bot request my URL?”

But:

- Was it training, search, retrieval, or user-triggered?
- Was my content cited or mentioned?
- Was it shown as a source?
- Did any traffic come back?
- Was there any measurable value for the publisher?

Without something like that, I don’t see how WordPress publishers are supposed to make a rational decision.

Blocking everything is a panic button. Allowing everything is blind trust.

There should be something in between. What is your opinion?

A discussion with friends and acquaintances made me think about AI bots and WordPress sites. They told me that Google, OpenAI, Anthropic and others run bots that crawl websites, partly for AI training, search, retrieval, or whatever else happens behind the scenes. They also said that allowing these bots might become important if you want your content to be mentioned, cited, or used in AI answers.

But from a normal WordPress site owner’s perspective, I see a basic problem:

How would I even know?

WordPress itself does not show me whether GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot or similar bots visited my site. Analytics usually shows human visitors and referral traffic, not raw crawler requests.

Many WordPress users also do not have easy access to raw server logs. And even if they do, they would still need to know which user agents belong to which AI crawlers and what those requests actually mean.

But the bigger question for me is this:

Even if I could see that an AI bot requested one of my URLs, how would I know whether my site was ever mentioned, cited, used as a source, or included in an AI answer?

With Google Search, we have Search Console.
With visitors, we have analytics.
With WordPress, we have plugins for almost everything.

But is there anything practical for AI crawler visibility or AI mentions?

Are WordPress site owners currently tracking this somehow, or are we basically blind unless we have server-log access and know how to interpret AI crawler requests?

>The view count suggests people are at least curious, but the lack of concrete answers suggests the tracking side is still unclear.

[Update]

That makes me wonder how SEO people are supposed to measure this reliably. With classic search there is at least some kind of chain: crawling, indexing, rankings, impressions, clicks. With AI answers, I don’t see the same chain yet. If crawler visits, mentions, citations and referrals are all separate signals, how do we know what actually worked?

In my previous post, I asked why WordPress sites should allow AI crawlers when they create load, take content and send little traffic back:
https://www.reddit.com/r/Wordpress/comments/1sxvtqz/wordpress_sites_why_allow_ai_crawlers_if_they/

AI crawlers feed chatbots. AI companies make money. Publishers get two bad choices:
block them, or hope.

Are you already blocking - or just hoping?

Do you know how much they scrape?

Google is the hardest case: the AI purpose is masked behind normal Googlebot activity.

OpenAI and Anthropic are different. Their bot descriptions help, but are too vague for real selective blocking. Blocking only by bot name may even remove useful signals.

Would you want to decide by bot, URL and request intent - instead of blindly blocking whole crawler families?

In my previous post, I asked why WordPress sites should allow AI crawlers when they create load, take content and send little traffic back:
https://www.reddit.com/r/Wordpress/comments/1sxvtqz/wordpress_sites_why_allow_ai_crawlers_if_they/

AI crawlers feed chatbots. AI companies make money. Publishers get two bad choices:
block them, or hope.

Are you already blocking - or just hoping?

Do you know how much they scrape?

Google is the hardest case: the AI purpose is masked behind normal Googlebot activity.

OpenAI and Anthropic are different. Their bot descriptions help, but are too vague for real selective blocking. Blocking only by bot name may even remove useful signals.

Would you want to decide by bot, URL and request intent - instead of blindly blocking whole crawler families?