Over the past few months, llms.txt has become one of those things that shows up in every SEO newsletter, every LinkedIn post, and eventually in a message from a client asking why their site doesn't have one.
The pitch is appealing. Add a single Markdown file to your root directory, and AI systems like ChatGPT, Claude, and Google's AI Overviews will understand your site better and cite you more often. It sounds like robots.txt for the AI era.
There's now enough evidence to answer the question properly. The short version: for the vast majority of businesses, llms.txt is not worth your time. Google has said so directly, and the crawl data backs it up.
What llms.txt Was Supposed to Do
The llms.txt specification proposes a Markdown file at your domain root — yoursite.com/llms.txt — containing a curated summary of your site with links to your most important pages.
The reasoning behind it is sound. Large language models work within a limited context window. HTML pages are full of navigation, scripts, cookie banners, and markup that waste that budget. A clean Markdown summary would, in theory, let an AI system grasp your site efficiently.
It's a genuinely reasonable idea. The problem is that nobody with a major AI crawler agreed to read it.
What Google Actually Said
This is where the case falls apart. Google's AI optimisation guidance, updated in June 2026, states it plainly:
You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them.
That's not ambiguous. Google's own AI-features documentation lists llms.txt among tactics you do not need for AI Overviews or AI Mode.
Gary Illyes said at Search Central Live in July 2025 that Google does not support llms.txt and has no plans to. When John Mueller was asked whether Google publishing its own llms.txt file counted as an endorsement, his answer was about as direct as Google gets: "I'm tempted to say something snarky since this has come up so often, but to be direct, no."
Mueller also drew a comparison worth sitting with — he likened llms.txt to the old keywords meta tag: a file where a site declares what it's about, when the search engine would rather just read the site and decide for itself.
Anyone who was doing SEO in 2009 knows how that story ended. Self-reported relevance signals get ignored, because they're trivially gamed.
The Crawl Data Is Worse Than the Statements
Official positions are one thing. Actual bot behaviour is another, and it's more damning. An analysis of 137,000 websites found:
| Finding | Result |
|---|---|
| Sites that had published an llms.txt file | 28% |
| Of those files, how many were never requested by any bot | 97% |
Read that second row again. Nearly every llms.txt file that exists has never been fetched — not by Google, not by OpenAI, not by Anthropic, not by Perplexity.
These aren't files that got crawled and ignored. They were never opened at all. A quarter of the web built a door that nobody has knocked on.
If you want to check this on your own site, don't take anyone's word for it. Open your raw server access logs and search for requests to /llms.txt. On the sites we've looked at, the count is almost always zero. That's a five-minute check, and it's more persuasive than any blog post — including this one.
So Why Does Everyone Keep Talking About It?
Three reasons, and none of them are "it works."
It's easy to sell. "Add this file" is a concrete, cheap deliverable. It looks like progress on an invoice. Real AI visibility work — earning citations, building genuine subject authority, getting mentioned on sites that models actually train on and retrieve from — is slow and much harder to package.
Tools shipped support for it. Yoast added an llms.txt feature. Various generators appeared. Once tooling exists, adoption follows, and adoption gets mistaken for effectiveness. The file being easy to create says nothing about whether it's read.
Correlation gets misread as causation. Sites that publish llms.txt tend to be run by teams who follow SEO closely and do a lot of other things well. When those sites show up in AI answers, the file gets the credit that belongs to the underlying content quality and authority.
What Actually Influences AI Visibility
The uncomfortable answer is that it looks a lot like good SEO — which is roughly what Google has been saying all along.
Ahrefs recently studied 331,000 pages and reached a conclusion worth repeating: Google doesn't punish AI content, it punishes bad content. The same principle applies here. There's no file that substitutes for being genuinely worth citing.
What does move the needle:
- Content that answers a question completely and can be quoted. AI systems extract passages. Clear, self-contained, factually accurate sections get pulled; rambling ones don't.
- Being mentioned elsewhere. Models retrieve from and are trained on the wider web. Getting cited on sites that already have authority matters far more than anything in your root directory. This is digital PR and content strategy work, not a technical checkbox.
- A site that renders fast and clean. If your key content only appears after heavy JavaScript execution, retrieval systems may never see it. This is ordinary technical site quality, and it was already worth fixing.
- Structured data. Unlike llms.txt, Schema.org markup is documented, actively consumed, and used by Google. If you want a machine-readable file that genuinely gets read, this is the one.
- Clear, consistent entity information. Who you are, what you do, and where — stated the same way across your site and off it.
None of that is new. That's rather the point. The teams doing well in AI search are largely the teams that were already doing the fundamentals properly.
The Honest Recommendation
Skip it — for now.
If you have a spare ten minutes and a developer already in the codebase, adding an llms.txt file is harmless. It won't hurt your rankings. Google has said it's "fine," which is a long way from saying it's useful.
But if creating and maintaining one would take real hours, displace other work, or appear as a line item on an agency invoice — that's a bad trade. You'd be maintaining a second copy of your site structure that must be kept in sync, in exchange for no demonstrated benefit.
The exception: if you run developer-facing documentation, there's a real case. Tools like Cursor and Claude Code can be pointed at documentation directly, and some documentation platforms have adopted the format meaningfully. If your users are developers who will actively feed your docs to an AI coding assistant, llms.txt may earn its keep. That's a genuine use case — it's just not the one being marketed to most businesses.
Revisit if this changes: if OpenAI, Anthropic, or Google announce official support, the maths changes overnight. It's worth watching. It's not worth pre-emptively building for.
Frequently Asked Questions
Does Google use llms.txt?
No. Google's AI-features documentation, updated June 2026, states that Google Search does not use machine-readable AI text files, and Gary Illyes confirmed there are no plans to support it. Google publishes its own llms.txt file, but John Mueller has explicitly said this is not an endorsement.
Will adding an llms.txt file hurt my SEO?
No. It's a static file that search engines ignore. There's no ranking penalty. The cost is the time spent creating and maintaining it, not any risk to your rankings.
Do ChatGPT or Claude read llms.txt?
There's no confirmed automatic support from any major AI provider. The crawl data is the clearest signal here — in a study of 137,000 sites, 97% of published llms.txt files were never requested by any bot at all.
What should I do instead to show up in AI answers?
Focus on content that's genuinely quotable and complete, earning mentions on authoritative third-party sites, fast clean rendering, and Schema.org structured data. These are documented, actively consumed signals — unlike llms.txt.
Is llms.txt the same as robots.txt?
No. robots.txt is a long-established standard that crawlers genuinely obey and that controls access. llms.txt is a proposed format that suggests content, and major AI crawlers have not adopted it.
The Takeaway
llms.txt is a sensible idea that hasn't been adopted by the systems it was designed for. Google has said it doesn't use it. The crawl logs show almost nobody else does either.
Being cited by AI systems comes from the same place it always has: being genuinely useful, clearly structured, and referenced by other people. There's no file you can upload that shortcuts that.
If you're unsure where your site actually stands in AI search — as opposed to where a checklist says it should — get in touch. We'll look at your real server logs and tell you honestly what's worth fixing.