Not all AI systems read and obey robots.txt
What do you think about this experiment / article? See https://www.searchengineworld.com/do-ai-assistants-actually-respect-your-robots-txt-we-put-12-of-them-to-the-test
u/AndreAlpar — 2 days ago
What do you think about this experiment / article? See https://www.searchengineworld.com/do-ai-assistants-actually-respect-your-robots-txt-we-put-12-of-them-to-the-test