r/TechSEO

[removed]

Hi everyone, I have enough budget to hire an Indian agency to seo 100 of my niche sites, they do an alright job, 7-10% of the sites will end up working out (I have been working with them for 2 years now).

The sites are targeting different niches (no high risk). (Courses, mock tests, notion templates, work certificates …).

I will be generating AI content on the sites & I’ll be obfuscating class names.

There won’t be any backlinks between these sites.

I don’t want Google to ban all of the sites, so what would be the best way to approach this?

Here’s my plan:
all sites will have unique AI content with surferseo keywords, the sites will have a few unique AI generated images on as well
All sites all the domains will be purchased in the same 3-4 weeks span & connected to the same cloudflare account (so same 2 unique nameservers for the 100 sites)
On all of the sites I will be using the same rybbit tracking stripy hosted on my private server.

I feel like the setup is a bit risky and that I need to spread the sites across multiple registrars, cloudflare accounts, hosting providers, different gsc accounts, and use different analytics scripts hosted on diff servers.
I’m just unsure if I’m supposed to go down this extra mile because it’s too much of a headache & ill likely need to hire someone to manage the complicated setup as well.

so after seeing all the "are AI bots ignoring robots.txt" threads i wanted to actually test it instead of guessing from logs.

the problem with guessing is anyone can set any user agent, and a lot of these fetchers don't come from published ip ranges. so every AI got its own secret url, nothing linked to it anywhere, meaning whatever hit that url had to be that AI. cloudflare worker in front logging everything. then in a fresh chat i asked each one to open the link and tell me a code that's on the page.

claude checked robots.txt before touching anything on a new site. when i disallowed the page it just refused and said it wouldn't try to get around it. kinda polite honestly

chatgpt was the interesting one. first visit to a brand new site it went straight to the disallowed page without reading robots.txt. about 76 seconds later OAI-SearchBot shows up and reads robots.txt. next time i asked, chatgpt refused the page without even making a request. so it learns the rules after the first visit lol

gemini comes in with a user agent that's literally just "Google" and never read robots.txt once. the ips weren't on any of google's published lists either

grok is where it got weird. every request, 9 to 12 different ips hit the page, most within a few seconds, all claiming to be chrome or safari. hosting companies, small isps in brazil, mexico, poland. a couple came through a network registered to oxylabs (the proxy company). 59 ips over 6 requests and it never reused one. never touched robots.txt

perplexity only sent its crawler to read my robots.txt and homepage, then told me the link might be phishing

to be fair openai and google both say fetches a user asks for can skip robots.txt, so those two are doing what they say. and it's a small test, few runs each, one day. but blocking by user agent clearly only stops the bots that tell you their name

also the first time i ran this i opened all the links in my own browser before pasting them, so the only visitor in my logs was me. took me a minute to figure out why every AI apparently used brave on windows

anyone else seeing bursts of chrome/safari traffic from random isps right after someone asks grok about a page?

so after seeing all the "are AI bots ignoring robots.txt" threads i wanted to actually test it instead of guessing from logs.

the problem with guessing is anyone can set any user agent, and a lot of these fetchers don't come from published ip ranges. so every AI got its own secret url, nothing linked to it anywhere, meaning whatever hit that url had to be that AI. cloudflare worker in front logging everything. then in a fresh chat i asked each one to open the link and tell me a code that's on the page.

claude checked robots.txt before touching anything on a new site. when i disallowed the page it just refused and said it wouldn't try to get around it. kinda polite honestly

chatgpt was the interesting one. first visit to a brand new site it went straight to the disallowed page without reading robots.txt. about 76 seconds later OAI-SearchBot shows up and reads robots.txt. next time i asked, chatgpt refused the page without even making a request. so it learns the rules after the first visit lol

gemini comes in with a user agent that's literally just "Google" and never read robots.txt once. the ips weren't on any of google's published lists either

grok is where it got weird. every request, 9 to 12 different ips hit the page, most within a few seconds, all claiming to be chrome or safari. hosting companies, small isps in brazil, mexico, poland. a couple came through a network registered to oxylabs (the proxy company). 59 ips over 6 requests and it never reused one. never touched robots.txt

perplexity only sent its crawler to read my robots.txt and homepage, then told me the link might be phishing

to be fair openai and google both say fetches a user asks for can skip robots.txt, so those two are doing what they say. and it's a small test, few runs each, one day. but blocking by user agent clearly only stops the bots that tell you their name

also the first time i ran this i opened all the links in my own browser before pasting them, so the only visitor in my logs was me. took me a minute to figure out why every AI apparently used brave on windows

anyone else seeing bursts of chrome/safari traffic from random isps right after someone asks grok about a page?

I built a Wordpress website for my niche mortgage business and used Claude to help me with SEO and AEO. I started building and indexing pages with the help of ChatGPT about 7 months ago and then about 3 months ago I connected Claude Code to my webpage and it's been managing my site since then. Mostly just making sure all the plugins are updated and making site wide changes I ask it to.

Here are some stats for the last 12 months and seems to be getting worse.

Month	Impressions	Clicks	CTR	Avg Pos
Nov 2025	139	9	6.47%	11.6
Dec 2025	462	3	0.65%	5.4
Jan 2026	376	10	2.66%	7.0
Feb 2026	2,333	14	0.60%	25.0
Mar 2026	5,911	22	0.37%	31.6
Apr 2026	4,951	15	0.30%	32.1
May 2026	3,453	8	0.23%	43.0
Jun 2026	4,218	15	0.36%	44.2
Jul 2026	11,859	19	0.16%	51.0
Aug 2026	11,138	9	0.08%	59.7
Sep 2026	3,488	6	0.17%	49.6

February/March is when I added 31 pages and ran FB ads. June is when I connected Claude Code and got rid of some low performing pages.

Here's just the high level stuff Claude has done or helped me do.

• Full technical audit + cleanup — fixed canonicals, broken links, sitemap, redirects
• Merged thin/duplicate pages into stronger hub pages (301 + noindex)
• Rewrote every title/meta; made each location page unique (no templated content)
• Added server-side schema (FAQ, LocalBusiness, Org/Person, reviews) so Google and AI can actually read it
• AEO work: entity + E-E-A-T signals, structured FAQs, llms.txt for AI answer engines
• Local SEO: optimized Google Business Profile, fixed citations/NAP across directories
• Off-site push: directories, reviews, and digital PR for backlinks
• Weekly GSC/GA4 tracking, plus monitoring AI-assistant referral traffic

I've been getting a solid amount of leads from LLMs which is awesome but I feel like my business would explode if I could start ranking on page 1. So what does it take to rank on page 1?

AI is useful, but it can’t replace real experience and industry knowledge. Founders and CEOs should use AI to support their teams, not replace specialists who understand the work beyond just generating answers.

What do you think? Has AI actually replaced experts in your company, or has it just become another tool?

Been a month and google has not indexed any other page of my site expect the home page.

I have tried all the usual things like proper sitemap, internal links, request indexing, proper prerendered html content, etc.

But all of those pages were kind of written a lot from ai, follow a general layout like a drop zone, 4-5 text cards and some faqs.

I think google is now tired of several similar vibecoded sites popping up and has learned the pattern of them quiet well that the crawler is not interested at all.

So I am thinking of replacing this with my own human written content, actual in depth content about that specific service and a more unique way of presenting the content.

Would that work?

Btw the site is called pixalgo, and the pages i am most interested in getting indexed are the services/ route pages.

Any help or comment would be appreciated. Also im using react router v8 vite prerendering.

We currently use a consultant, but management is pushing me to automate what they do and I'm not sure where to begin.

Hi

I recently seen my domain's Organic sessions is totally equal to engaged sessions, which results in 100 % engagement rate and "0" bounce rate.

I have checked real time, debug, GTM but found everything fine.

But can't able to figure out the issue, can anyone help me with this.

I need to know when organic sessions is same as engaged sessions which is not possible.

Where to check and what to sort?

Thanks in advance.

https://preview.redd.it/ys6bydgxeguh1.png?width=1496&format=png&auto=webp&s=6b4055ba20e170c4fcd7e0510157e580a68a5ccc

I'm very interested in your opinion. To me, this looks like a really huge mistake. I would never go to an SEO agency that can't calculate properly.

Hey everyone, I built indiedex.gg which is a search engine and recommendation engine to find steam games that normally don't surface on the steam store.

The way it works is that I consume all steam data, and enrich the data with social signals about each game and game studio. Then I do some data processes to make really good similarity and game suggestions.

Each game becomes a game details page and a games-like page, and each page supports 4 languages, currently my catalog size is 120k, so 120K * 2 pages * 4 languages = 960K pages.

However each page does provide unique value, which is the programmatic analysis of the game based on my data processing. For example, the game detail page will show this section:

https://preview.redd.it/z06cox432euh1.png?width=922&format=png&auto=webp&s=48cc0498ea89f07ffb294a5bf318990d8316a652

which is unique per game, and based on not only the data from social signals but also my system's generated tags for each game.

https://preview.redd.it/no3q22ca2euh1.png?width=1568&format=png&auto=webp&s=4da31708da1cccf065de90abfd2f23d4efab042a

and the games-like page does something similar but based on similarity signals and its own similarity analysis.

I understand google is cracking down on scaled content that it considers "not helpful", however, based on feedback from users and engagement data (average sessions are around 50 seconds, viewers visit 3.5 pages on average) I know the app is helpful and the people that love it, really love it.

I have seen a lot of different interpretations of what Google means when they say "scaled" and "helpful", but I don't really know what to expect here, this is the GSC chart, I launched this site on early August, and it seems the latest update resulted in a spike:

https://preview.redd.it/6hgfnoqa3euh1.png?width=2959&format=png&auto=webp&s=d1071959c9ffac7f5b1a2b980aa56ad26897e94e

I know some SEO, but I am not yet an expert, based on your experience, should I be shitting bricks and expecting Google to decide it is my time and then have it kill my site purely based on the fact that I generate pages programmatically?

EDIT:

A bit more information if it helps:

This is how many pages are indexed

https://preview.redd.it/2mi3baoi7euh1.png?width=1216&format=png&auto=webp&s=06c3fe1ae8e6e433ca6f3b75ae4cf85ca0185154

And the sitemap was submitted with ~21K pages

https://preview.redd.it/q59hqsjq7euh1.png?width=1152&format=png&auto=webp&s=6dd1841825b97d8a93a0c7adf147b2ef7258823a

https://preview.redd.it/y0tqmkf4vbuh1.png?width=1112&format=png&auto=webp&s=cb0a5064d55fe7336f260a01ebecd6d7c9ed7770

https://preview.redd.it/xr0fdph6vbuh1.png?width=1091&format=png&auto=webp&s=4897ed330ba5f10539858933db24f59d2ec67377

I’m a consultant who usually does the strategy and hands off the execution. I recently started my own small agency and this is the first site where I’m doing the full SEO work myself, so I’d really value a sanity check.

The site: a local healthcare-adjacent service business (court-ordered / compliance-related services) in a large US metro. The site is about 3 months old, it’s a WordPress site on Elementor, and I’m not running ads yet. I’m also managing the Google Business Profile.

Search Console (web only):

Last 28 days: 20 clicks, 589 impressions, 3.4% CTR, average position 20.1

Last 7 days: 6 clicks, 163 impressions, 3.7% CTR, average position 21.4

A few form submissions so far

What I’ve done so far:

Rebuilt the core service pages and added new dedicated pages for the main services (including one for the online version)

Rewrote titles, meta descriptions and headings around the target keywords, and cleaned up internal links and the menu

Added schema (local business, FAQ) plus a credentials/trust section that links to the official state listings

Removed the theme’s demo content, fixed 404s and submitted validation in Search Console

Speed work: CSS aggregation, a preloaded hero image, resized images. Desktop PageSpeed is about 91 and mobile is in the low 60s (heavy theme)

GA4 events for calls, directions clicks and form submissions, plus spam protection on the forms

Started on directory listings and citations (Yelp is verified, the rest are in progress)

Added a short explainer video on the main pages

Where I’m unsure:

For a niche like this, are ~590 impressions a month and an average position around 20 reasonable at 3 months, or should I be further along?

I don’t have Semrush or Ahrefs, so I can’t see full search volumes. What’s the best way to judge whether the keywords are worth chasing without those tools?

There’s a lot of spammy backlink junk pointing at the domain. I’ve held off on a disavow. Is leaving it alone still the right call?

What would you prioritize next: reviews and the Business Profile, links from local professionals, more content, or something I’m missing?

Platform question for future builds: is WordPress still the best choice for SEO and content-heavy local sites, or is there a CMS or platform you’d pick instead (Webflow, Shopify, a headless setup, something else)? The theme and page builder on this one are heavy, and mobile speed is the weak spot, so I’m curious what you’d choose for a small local business site in 2026 and why.

Happy to share more detail on the queries and pages if it helps. Thanks in advance, and be as blunt as you like.

My website has a sitemap index, which contains URLs to hundreds of sitemap files. And each sitemap file contains URLs to pages in my website. So for instance

Index: /sitemap/sitemap-index.xml

sitemap:

- /sitemap/products-1.xml (contains thousands of URL)

- /sitemap/products-2.xml (contains thousands of URL)

When I submit the sitemap index file into GSC, it is able to read it. But when I click on the record, turns out it only read 1 sitemap, which is the first entry in the sitemap index (which is products-1.xml). Thus it only discovers the URLs inside products-1.xml, while the rest of URLs inside other sitemaps are not discovered.

Im not sure whether anyone experiences this issue? How to fix it? Is GSC buggy?

My hypothesis is that non-human requests can be identified by non-human review with a good collection strategy. This is how five days worth of data was interpreted by claude:

1,849 records, 181 distinct user agents

• User-initiated fetches.
  • ChatGPT-User fetched two pages (5 October 19:17 and 6 October 17:30), with no referer. One minute after the first, OAI-SearchBot began a 30-path crawl that lasted an hour.
  • Google-NotebookLM made one fetch on 4 October.
• ClaudeBot. It made 59 requests, all to robots.txt and sitemap.xml (2 distinct paths), and never fetched a page.
• Meta. 190 requests are almost all with a referer.
• GPTBot. Seven requests, first seen 4 October 08:28.
• Bursts.
  • 277 requests in two minutes on 5 October at 00:42, from Chrome 131 on Linux, across 272 distinct paths, with no referer. That pattern fits a path scanner. The status column would confirm it.
  • 34 requests from Chrome 128 with OPR/114 in two minutes at 23:48 that night, across 32 paths.
  • 38 AhrefsSiteAudit requests in one minute (an audit run, separate from AhrefsBot).
• Scanners and profilers. Palo Alto (25), Censys (7), MerchantSecurityScanner (2), BuiltWith (2) and gitscan (2).
• Google tools.Google-InspectionTool made 9 requests and Googlebot appeared between 19:49 and 19:53 today. If you ran a live test in Search Console around 20:50 BST, those are yours (NB: I did do this).
• Stale browser strings. Chrome 48, 57, 76, 80, 85, 89, 99 and 113 appear.

This is a new feature of https://clientlog.bayis.co.uk which already captures client-side requests from privacy respecting beacon script.

We found a product page where the structured data was valid, but key product facts and their relationships were buried deep inside a nested u/graph structure.

The data existed.

Passing validation didn't automatically mean every consumer would interpret those relationships as intended.

That distinction matters for ecommerce:

• Which price belongs to which variant?
• Is the displayed price recurring or one-time?
• Is availability attached to the right SKU?
• Do reviews belong to the product or the whole category?

A green validation result answers only part of the question.

The real test is whether the information is clear, consistent, and correctly associated with the product.

Hot take: too many technical SEO audits stop at “valid schema” instead of checking whether the data tells the right story.

SEO folks: what do you use to test actual structured-data interpretation beyond a validator?

I recently joined a new organization and one of my responsibilities is managing their Google analytics. In a GA4 roll-up (consolidating the main domain and 2 subdomains) property, I'm seeing rows where New users is much higher than Total users, and sessions per total user is way past anything a real person would do. I haven't been able to pin down why. We use a blended reporting identity. Here's what I'm seeing below reviewing a week over week campaign performance.

https://preview.redd.it/mrfpt5ta8kth1.png?width=2020&format=png&auto=webp&s=0740563706ce767abc69db354425c9ced3d8ddad

Google's docs say new users can exceed active users but shouldn't exceed total users, and that thresholding can cause discrepancies.

My question is, what would you recommend I check to diagnose this issue. Has anyone else seen this issue in GA4? Could it be a consent mode issue? Is there a way to tell thresholding from modelling when the data quality icon isn't showing anything?