Does llms.txt Help SEO? Google Says It Ignores the File
Google Search ignores llms.txt and 97% of published files get zero requests. The evidence, who should still ship one, and what moves AI visibility instead.
Long Nguyen
Founder & Research Lead
Short answer: no. Google Search ignores the file, and server-log studies show almost nothing else reads it either. But does llms.txt help SEO is the wrong question to stop at, because the file was never proposed as a search feature — and the version of the spec published this month changes what it is for. Here is the evidence, the narrow case where the file still earns its place, and what to do with the hours you would have spent maintaining it.
What Google actually says about llms.txt
On Google Search Central published Optimizing your website for generative AI features on Google Search, its first consolidated AI-optimization guide. It contains a section titled Mythbusting generative AI search, and llms.txt is the first item on the list of things you can ignore.
One month later, on , Google added a clarification note to that same guide. The changelog entry is called Clarifying guidance on llms.txt files, and the note says that maintaining one will neither harm nor help your visibility or rankings in Google Search, because Google Search ignores them.
That wording is worth reading precisely, because most coverage blurred it:
- Ignored is not the same as never fetched. Google's guide notes that Googlebot may discover, crawl and index many kinds of files beyond HTML. Your llms.txt can end up in the index like any other text file. What it does not get is special treatment.
- Ignored is not a penalty. There is no downgrade for having one. The file is neutral, not risky, from a ranking standpoint.
- The clarification narrowed the claim, it did not reverse it. Earlier guidance discouraged the file broadly; the June note scopes not needed to Google Search specifically while conceding that other services may use it. Friendlier phrasing, identical substance.
None of this was new in 2026. The position has been consistent for over a year:
| When | What Google said |
|---|---|
| July 2025 | Gary Illyes, at Search Central Live: Google does not support llms.txt and is not planning to. |
| July 2025 | John Mueller suggests applying noindex to the file, since it can get indexed and looks odd to users who land on it. |
| May 2026 | The AI optimization guide lists llms.txt under Mythbusting — things you can ignore for Google Search. |
| June 2026 | Docs clarification: keeping one neither harms nor helps, because Search ignores it. Mueller calls the file speculative on the record. |
If you have been told Google softened its stance, the softening was tonal. The ruling did not move.
Do any AI systems actually read llms.txt?
Google's position only covers Google. The more useful question is whether anything fetches the file — and unlike most AEO claims, this one is measurable from server logs.
Ahrefs published the largest study to date on , covering 137,210 domains in its Web Analytics dataset and every request to /llms.txt across that population during May 2026:
| Finding | Number | What it means |
|---|---|---|
| Domains publishing a valid llms.txt | 28% | Adoption is real. Treat it as an upper bound — Ahrefs customers skew technical. |
| Valid files that got zero requests in May | 97% | No bots, no humans. Roughly 1,100 of ~38,000 files saw any traffic at all. |
| Of the traffic that did occur, share from bots | 96% | The 4% human slice is largely SEOs previewing links in chat apps. |
| Share of fetches from named AI tools | 19.5% | GPTBot first, Claude-Code second — ahead of every AI search and assistant bot. |
| Share of fetches from GEO/AEO tools, checkers and researchers | 12% | Part of the llms.txt economy is tooling auditing compliance with a convention nobody consumes. |
| Requests for llms.txt on domains that do not have one | Zero | AI bots never probe for the file. They are not looking and failing — they are not looking. |
That last row is the one that settles the argument. A crawler that wanted the file would ask for it and collect 404s. None of them ask.
A separate SE Ranking analysis across roughly 300,000 domains modelled whether having an llms.txt correlated with how often a domain was cited in AI answers, and found no statistically significant relationship — removing the variable improved the model. Two independent datasets, two different methods, same conclusion.
The one genuine signal inside the noise: Claude-Code outranked every AI retrieval bot. The file's real audience is coding agents pulling documentation at build time, which is exactly the use case Jeremy Howard proposed it for in 2024. It is not, and has never been, a search-visibility mechanism.
So does llms.txt help SEO? A claim-by-claim verdict
Help is doing a lot of work in that question. Split it into the four things people actually mean:
| The claim | Evidence | Verdict |
|---|---|---|
| llms.txt improves Google rankings | Google's docs state Search ignores the file | No. Officially and explicitly no. |
| llms.txt gets you into AI Overviews or AI Mode | Same guidance: no special file, markup or Markdown is needed or used | No. Eligibility comes from being indexed and snippet-eligible. |
| llms.txt increases citations in ChatGPT, Claude or Perplexity | No provider has committed to reading it; retrieval bots barely fetch it; no measured correlation | No evidence. Not disproven in every case, but unsupported. |
| llms.txt makes agent retrieval cheaper and more accurate | Coding agents do fetch it; token savings from clean Markdown are large and documented | Yes, narrowly. The benefit accrues to the agent, on sites agents already visit. |
The honest framing is that llms.txt is plumbing for agents, not a growth lever for you. If the pitch you were sold used the words ranking, citation or visibility, you were sold the wrong file. If you want to see the format itself before deciding, our annotated llms.txt examples and copy-paste template walks through real files element by element.
Can publishing llms.txt hurt you?
Not in rankings. But neutral is not the same as free, and four real costs get skipped in the enthusiasm:
- Drift. A stale file is worse than no file. If an agent does fetch it and half the links 404 or describe pages you rewrote a year ago, you have handed it confidently wrong context about your own site. Hand-maintained files rot fastest.
- It can land in search results. Because Google may index it like any text file, a linked llms.txt can surface in the SERP and confuse a human who clicks it. Mueller's own suggestion is to noindex it — set
X-Robots-Tag: noindexon the response rather than trying to use a meta tag in a plain-text file. - You are publishing a curated map of your best pages. That is the point, and it is also a gift to competitors and scrapers. Nothing in llms.txt is private, and listing a page grants no permission and blocks nothing — access control still lives in robots.txt.
- Opportunity cost. This is the expensive one. Hours spent curating a file that 97% of the time nobody requests are hours not spent on the content and crawl work that does move AI visibility.
Who should still publish an llms.txt file
The decision is not ideological, it is about cost of maintenance versus likelihood of consumption:
| Site type | Publish one? | Why |
|---|---|---|
| Developer tools, API products, public docs | Yes | Coding agents are a real distribution channel here. Claude Code and Cursor fetch docs live, and your docs platform probably generates the file already. |
| SaaS with a documentation site | Yes, generated | Cheap if automated. Never hand-maintain it. |
| eCommerce aiming at shopping agents | Optional, low priority | Cheap to ship, but no measured payoff yet. Fitment, pricing and returns data in the descriptions is the only version worth writing. |
| Local business, service site, small marketing site | No | Nothing in your visitor mix reads it. Spend the hour on content or on crawl access. |
| News and media | No | High churn guarantees drift, and your value is in the articles, not a table of contents. |
The operating rule: publish it only if a script generates it. If shipping it costs a build step and near-zero upkeep, the expected value is small but positive. If it costs a recurring human hour, it is negative. Our free llms.txt generator will scaffold a first draft from your existing pages so you can judge the file rather than the idea — and if you want the generate-then-automate path in full, see how to generate an llms.txt file.
What to do instead if you want AI visibility
Google's own guide is unusually specific about where the leverage is, and it is not in files:
- Be indexable and snippet-eligible. A page cannot appear in AI Overviews or AI Mode unless it is indexed and eligible to show with a snippet. This is the floor, and it is where most real problems hide.
- Do not block the crawlers you want citations from. The most common cause of AI invisibility is an over-tight robots.txt or bot rule. Check what you are actually allowing — our guide to robots.txt for AI crawlers covers the user-agent list.
- Write non-commodity content. Google's example contrast is instructive: 7 Tips for First-Time Homebuyers is commodity; a first-hand account of waiving an inspection is not. AI systems reviewing many sources favour the one with a point of view.
- Stop chunking and rewriting for machines. Google states plainly that content does not need fragmenting into tiny pieces and does not need AI-specific phrasing. Both tactics are still being sold.
- Measure it. Search Console has a Generative AI performance report. Use your own data instead of a third-party tool claiming access to internal Google metrics — no such access exists.
Notice that four of those five are ordinary SEO. Google's framing is that optimizing for generative AI search is optimizing for search, which is why it treats AEO and GEO as labels rather than disciplines. The engineering half — crawl access, structure, agent-readiness — is real work, but it sits alongside content quality, not instead of it.
Where llms.txt goes from here
The proposal is not standing still, and the change is recent enough that most guides published this year are now describing an outdated file. On , Jeremy Howard published v2 of the llms.txt spec. Four changes matter:
- The file can live at any path, not only the domain root.
/docs/llms.txtcovers everything under/docs/, and where several apply, the most specific wins. This lets sites that control only a subpath participate. - Discoverability got an answer. v2 adds standard link relations:
rel='alternate' type='text/markdown'points at a page's Markdown version, andrel='describedby'points at the llms.txt covering it. Both work as HTML<link>elements or as an HTTPLink:header, which means a CDN can add them without touching a single page. - Markdown URL forms both count. v1 specified
page.html.md; v2 also blessespage.md, matching what publishing tools already do. - The Optional section lost its mechanical meaning. Context-expansion tooling is no longer part of the proposal, so Optional is now just a convention for secondary links. If your file or your guide still treats it as a machine-readable instruction, that is a v1 assumption.
Meanwhile Chrome's Lighthouse audits for llms.txt under agentic browsing — which is where a lot of the do I need this anxiety comes from — while Google's own AI guide points readers instead toward agent-friendly page design and emerging protocols for agents that act rather than read. That is the more interesting direction: a static description file is a weaker primitive than a live interface an agent can call.
Our read for the rest of 2026: llms.txt stays useful for documentation, the v2 link relations are the genuinely new idea worth implementing, and anyone selling it as an AI-ranking tactic is selling a file, not a strategy. If you want the visibility work done properly — crawl access, indexing, structure, agent-readiness and the content that actually earns citations — that is what our SEO, AEO and GEO service covers end to end.