Does the llm.txt file really work?
And other AI LLMS text file questions answered for deep insight.
TL;DR
Yes, llms.txt does work — but only when you build all four LLMS files (llms.txt, llms‑ctx.txt, bing‑llms.txt, google-ai-overviews.txt) and follow a deliberate process. Most advice online is incomplete or flat‑out wrong. I spent days refining each file with the top AI models and documented what actually matters: company‑first structure, authority signals, consistent sections, and letting each AI model grade and improve its own file. This post answers the most common LLMS questions and gives you a practical section list you can use to build a best‑in‑class llms.txt for your own site.
Why I wrote this
Most LLMS.txt advice online is either incomplete or flat‑out wrong. After building all four LLMS files for a large, complex site, I wanted to share what actually works so others don’t have to spend days figuring it out.
Hi, fellow Redditors. I spent many hours/days working with the top 4 AI models to create near‑perfect LLMS text files for them. So I thought I’d share insights and answers to the most common questions, so others won’t have to work as hard. First, a disclaimer for SEO professionals thinking, “Hours? LLMS.txt is easy — shouldn’t take that long.” You’re correct if you have a small website with a couple dozen pages and one or two products. But when you’re dealing with a large site with hundreds of pages, diverse offerings, a unique business model, and a 30‑year company history… let’s just say you learn a lot about how to make LLMS files actually work. 🤪 Let’s start with the main question.
Does the llm.txt file really work?
Yes — especially for companies with large or complex websites. For smaller sites, it’s more about preparing for future needs. But regardless of size, LLMS files only work if done properly, and the process I used helps ensure they actually impact AI visibility. And when I say “yes, it works,” I’m referring to implementing all four LLMS files, not just llms.txt. Context is everything. You may have seen headlines like “Analysis of 300,000 domains shows llms.txt doesn’t impact how AI systems see or cite your content.” Catchy title — but dig deeper. Only ~30K sites even had llms.txt. No breakdown of:
- site size
- industry
- complexity
- number of offerings
- whether the file was implemented correctly
- whether the other three LLMS files were used
All of those matter. That article could be summarized in one sentence: About 10% of sites implemented llms.txt, and the other three LLMS files weren’t mentioned at all. Below are answers to the most common questions to help you understand the reasoning behind my approach.
What is an llms.txt file?
Contrary to what many SEO professionals say, llms.txt is not “a curated guide to your website.” It’s a curated Markdown guide to your company, designed to help AI agents understand your trusted content.
Is llms.txt a real thing?
Yes — it’s a new standard in the making, similar to how robots.txt started. But llms.txt is only ¼ of the system. You should create all four files:
llms.txt
bing-llms.txt
llms-ctx.txt
google-ai-overviews.txt
And no — llms-full.txt is not worth the effort. AI token limits mean they often only digest the first ~40 lines of llms.txt anyway. Sitemaps and robots.txt already cover full URL lists. llms-full.txt is something to revisit next year.
Does adding an llms.txt file improve SEO or AI visibility?
Not directly for traditional Google rankings. But it does increase the likelihood that AI models will accurately cite your company and understand your offerings. For large sites or sites with weak SEO in certain areas, LLMS files can help AI discover content it previously missed. There’s also a crossover effect between AI models.
Do major AI systems or Google use llms.txt?
Yes — especially when you implement all four files and keep them consistent and best‑in‑class. Here are the scores our files received (1–10), rated by each AI model:
- OpenAI LLMS: 9.8 (llms.txt)
- Bing/Perplexity LLMS: 9.9 (bing-llms.txt)
- Claude LLMS‑CTX: 9.5 (llms-ctx.txt)
- Google AI Overviews: 10 (google-ai-overviews.txt)
Google’s file is tiny and easy, but surprisingly, many don’t reach a score of 10.
What is the biggest and most common failure when creating llms.txt files?
Designing it with a website‑first approach instead of a company‑first approach. I started with “What do I want AI to know about our website?” After many iterations — and after each AI model critiqued its own file — the structure shifted toward a company‑first approach. Sections like:
- company metadata
- structured data
- primary audiences
- competitive differentiators
- FAQ
- published work
- industry recognition
- intellectual property
- owned brands
- entity aliases
…all became essential.
Does llms.txt replace robots.txt or sitemap.xml?
No. LLMS files are for AI ingestion. Robots.txt and sitemaps are for search engines and scrapers. Different tools, different purposes.
How to make an llms.txt file
Create a new llms.txt file and describe your company and website using plain‑text Markdown. Upload it to your website root so AI crawlers can find it. Then add this rule to your robots.txt:
Allow AI crawlers to access LLMS files
User-agent: *
Allow: /llms.txt
See the section list below for guidance. If you want a link to a best‑in‑class example, ask in the comments. After your first draft, ask ChatGPT to score it from 1 to 10. Implement the suggestions, then ask again. Repeat until your score is 9 or higher. Note: Some websites (including ours) block ChatGPT’s user window from accessing the file directly. That does not mean the bot can’t access it — only public users. If that happens, just paste the file content into ChatGPT.
What should be in an llms.txt file?
An llms.txt file should give AI models a clean, factual snapshot of your company: who you are, what you offer, who it’s for, and why you’re authoritative. It’s not a marketing page — it’s a structured knowledge file. llms.txt is the master file. Build it first, polish it, then reuse its sections across llms‑ctx.txt, bing‑llms.txt, and google-ai-overviews.txt.
Example llms.txt section list (use what applies, skip what doesn’t)
1. Company name and aliases
- One‑sentence identity summary
2. Website root
- Main homepage URL
3. Company metadata
- Founding year, ownership, certifications, BBB rating, etc.
4. Structured data
- Delivery modes (online, offline, video, onsite)
- Types of offers (courses, software, services)
5. Last Updated
- Helps AI know the file is current
6. Primary audiences
- Who your products are designed for
7. Core products and services
- High‑level product categories
8. Search keywords
- Terms users associate with your brand or offerings
9. Competitive differentiators
- What makes your products different or better
10. Product relationships
- How your products connect or complement each other
11. “If user wants…” sections
- Helps AI map user intent to the right product
12. Categories (for large or diverse sites)
- Organized product or content groupings
13. Frequently Asked Questions
- Short factual answers to common queries
14. Authority pages
- About, reviews, certifications, trust signals
15. Published work and industry recognition
- Trade magazines, media features, long‑term credibility
16. Citations and academic references
- Papers, journals, research that reference your work
17. Intellectual property
- Trademarks, proprietary methodologies, patents
18. Product ecosystem (if applicable)
- Brands, sub‑brands, product families
19. Entity aliases
- Alternate names users or publications use
20. Preferred citation pages
- Pages AI should reference when summarizing your brand
21. Contact information
- Support, sales, general inquiries
Why these sections matter
- Authority pages → help AI trust your brand
- Product relationships → help AI recommend the right product
- Competitive differentiators → help AI explain what makes your offerings unique
- Entity aliases → prevent misclassification
- Preferred citation pages → reduce hallucinations
- Search keywords → help AI map user queries to your content
How to test llms.txt
Ask the AI platform for the file it is designed for. It’s no different than asking a customer for feedback — you’re explaining your company to each AI model in the way it understands best.
Are major AI crawlers actually reading llms.txt yet?
Yes — if you follow this guide. No — for everyone else. SEO professionals often answer from a technical standpoint rather than from intent. It’s true that AI models aren’t “seeking out” llms.txt the way robots.txt or sitemaps are. But that negative reply is based solely on llms.txt — not the other three files. Even without directly opening llms.txt, the process above gets your information into the AI’s knowledge base. And when all four LLMS files are consistent, AI models treat your content with higher credibility. The bottom line: you want AI to understand your business more fully and clearly. LLMS files help you do that.
Advanced FAQs (often asked by developers or security‑minded users)
These questions come from people with a web development or data privacy background. They’re not essential for building an effective llms.txt file, but they help clear up misunderstandings.
Can llms.txt be used to block AI scrapers or prevent training?
No. LLMS files are informational only. Use server‑level controls (like htaccess rules) to block specific bots.
Should llms.txt include only key pages or a full sitemap?
Only key pages. Think of it as a “best foot forward” file. Include the pages that define your brand, establish your authority, and outline your core offerings.
If you want me to share links to our best‑in‑class LLMS files or have a question, just ask in the comments.