AI Search Optimization

llms.txt Examples: Real Files, the Format, and What Actually Works in 2026

Real llms.txt examples with annotated code, the official format, llms-full.txt, a copy-paste template, and honest 2026 data on whether the file works.

Long Nguyen Avatar

Long Nguyen

Fullstack Developer · AI Engineer · Researcher

6 min read

llms.txt is a Markdown file at your domain root that hands AI systems a curated map of your most important pages. This guide walks through real, annotated llms.txt examples, the full format straight from the official spec, how it differs from robots.txt and sitemap.xml, a copy-paste template, and — the part most guides skip — the honest 2026 data on whether the file does anything for you or only for the model reading it.

What llms.txt is — and what it is not

Flow diagram showing how an AI agent fetches an llms.txt file at inference time to find and retrieve a site's key pages, including llms-full.txt, before composing a grounded answer.

llms.txt was proposed by Jeremy Howard of Answer.AI in September 2024, with the canonical specification hosted at llmstxt.org. It is a community convention, not a formal standard — there is no IETF or W3C ratification as of 2026, and no central authority enforcing it.

The idea is narrow: language models work with limited context windows, so instead of making a model wade through your whole site, you publish a short Markdown file that says here is who we are and here are the pages worth reading. Howard is explicit that this is an inference-time helper — a file the model fetches while answering a question — not training data and not a form of persistent memory. Nothing is stored on the model's side; it re-fetches the file each time it decides to use it.

One consequence matters more than the rest: the blockquote summary at the top is the single most load-bearing line in the file. AI systems tend to lift it verbatim as the canonical, one-sentence definition of your brand — some tooling effectively treats it like a system prompt that grounds everything else. In practice, whatever you write there becomes what the model treats as the truth about you. Write it factually; drop "leading" and "best."

Just as important is what llms.txt is not, because a lot of writing gets this wrong:

  • It is not robots.txt. robots.txt controls crawler access at crawl time. llms.txt is a content directory read at inference time. They coexist; one does not replace the other.
  • It is not a permissions or opt-out file. Listing a page grants nothing and blocks nothing. Access control still lives in robots.txt and AI-bot user-agent rules.
  • It is not a security mechanism. Everything in llms.txt is public by definition — never treat it as a private index.
  • It is not a ranking signal. Publishing it does not lift you in classic search or AI answers (the evidence is below).

The llms.txt format, element by element

The spec is deliberately minimal. The whole file is valid CommonMark Markdown, served as plain UTF-8 text. Only one element is strictly required; everything else is a recommended convention.

Element Required? What it does
H1 (#) Yes — the only required line Your site or brand name. The entity anchor everything below maps to.
Blockquote (>) Recommended One or two factual sentences defining what you do. The line LLMs treat as your canonical description.
Context prose Optional Free-form Markdown (no headings) with details on how to interpret the file.
H2 sections (##) Optional Groups of links, e.g. Docs, API, Products. Each item is - [name](url): notes.
H2 named Optional Optional Special meaning: links here can be skipped when context is tight.

A few rules separate a valid file from a broken one: use exactly one H1, use absolute https:// URLs only, make sure every link returns a 200, avoid JavaScript-only pages (the model fetches the HTML shell and gets nothing), and keep it curated — the practical 2026 guidance is 3–5 sections and roughly 10–20 links, not your full sitemap.

Real llms.txt examples you can learn from

The format reads best through examples. Here are three that cover the patterns you will actually ship.

The minimal valid file

An H1 is the only required element. This is the smallest file that validates — and because the blockquote is what gets cited, even this does real work:

# Acme Analytics
> Acme Analytics is a product analytics platform that helps SaaS teams track events, build funnels, and measure retention.

The docs / SaaS pattern (the most common one)

The dominant real-world pattern comes from developer-tools and documentation sites — Anthropic, Stripe, Vercel, Cursor, and the many Mintlify-hosted docs sites all ship a variant of this. Links are grouped by function, and low-priority pages sit under ## Optional:

# Acme Analytics
> Acme Analytics is a product analytics platform that helps SaaS teams track events, build funnels, and measure retention.

Acme's docs are the fastest path to a working integration. Start with the quickstart, then the API reference.

## Docs
- [Quickstart](https://example.com/docs/quickstart): Install the SDK and send your first event in 10 minutes.
- [Concepts](https://example.com/docs/concepts): Events, users, funnels, and retention explained.

## API
- [REST API](https://example.com/docs/api): Endpoints, auth, and rate limits with examples.
- [Webhooks](https://example.com/docs/webhooks): Event payloads and delivery guarantees.

## Optional
- [Changelog](https://example.com/changelog): Release notes and breaking changes.
- [Blog](https://example.com/blog): Product and engineering articles.

The commerce / non-docs pattern

Most articles only show docs examples, which is useless if you run a store. The same structure works — swap the sections for what a buyer or an AI shopping agent needs. For commerce, the useful move is to pack quick facts (categories, price ranges, fitment) into the descriptions so an agent answering "does this fit my car and what does it cost" can get the gist in one cheap fetch instead of crawling every product page:

# Northwind Parts
> Northwind Parts is an online auto-parts store selling OEM and aftermarket components for European vehicles, with fitment guides and next-day shipping.

Northwind covers 40,000+ parts. The pages below are the ones worth reading first.

## Catalog
- [Shop by Vehicle](https://example.com/vehicles): Find parts by make, model, and year.
- [Brands](https://example.com/brands): OEM and aftermarket brands, typical price ranges per category.

## Buying Guides
- [Fitment Guide](https://example.com/guides/fitment): How to confirm a part fits your car.
- [Returns](https://example.com/returns): 30-day returns and warranty terms.

## Optional
- [Blog](https://example.com/blog): Maintenance tips and how-tos.

Notice the pattern across all three: an honest one-line definition, then a handful of sections that answer real questions, then genuinely skippable links under Optional. That curation — not length — is the whole point.

llms.txt vs robots.txt vs sitemap.xml

These three files are constantly confused because they all sit at your root. They do different jobs and are meant to coexist:

  llms.txt robots.txt sitemap.xml
Purpose Curated content map for AI Crawler access control Complete URL inventory for indexing
Consumed at Inference time Crawl time Crawl / index time
Controls access? No Yes No
Format Markdown Plain-text directives XML
Curated or exhaustive? Curated (10–20 links) Rules, not links Exhaustive
Who reads it AI assistants / agents (opt-in) Search + AI crawlers Search engines

If my sitemap already lists every URL, why do I need llms.txt?

Mostly, you don't — and it's worth being blunt about why. Your sitemap already exposes every URL on the site, and the major AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) already crawl the open web much like search engines, extracting and indexing your pages whether or not an llms.txt exists. So llms.txt adds nothing to discovery. The URLs are already findable.

What llms.txt actually changes is efficiency at answer time. A rendered HTML page is mostly noise to a model — navigation, footers, cookie banners, scripts, tracking. Stripping a mid-sized site into clean text can balloon past 500k tokens, well over most context windows; a curated Markdown index plus clean text can deliver the same substance in roughly 2–10k tokens. That means faster, cheaper retrieval and fewer hallucinations for the model.

Read that carefully: the benefit accrues to the LLM, not automatically to your website. It is closer to a sitemap than to a growth lever — useful infrastructure that helps a model parse you efficiently if it chooses to use the file, but it does not make your content more authoritative or more likely to answer a user's question. That job still belongs to the content itself.

llms-full.txt: what it adds and when to bother

llms.txt is an index — a table of contents pointing at pages. Its companion, llms-full.txt, is the opposite: it concatenates the full Markdown body of those pages into one file, separated by --- dividers, served at /llms-full.txt. The point is a single fetch: an agent grabs one URL and has your entire relevant corpus in context, with none of the navigation, ads, or scripts a normal page carries.

When it earns its place: documentation and product content that AI tools pull at inference time. A clean Markdown bundle cuts the token cost of that retrieval and improves the accuracy of what comes back — Vercel is the example people cite most. When it does not: sprawling marketing sites where dumping everything into one file produces noise, not signal.

A common setup is to ship both — llms.txt as the curated index and llms-full.txt as the full body — keeping the index lean while the full file carries the depth. That is the two-file pattern we run on our own site, and it is the right default for anyone whose content is genuinely meant to be read by agents.

A copy-paste llms.txt template

Start from this, replace the placeholders, keep the sections to what actually matters, then validate:

# Your Brand Name
> One or two factual sentences: what you do, who you serve, what makes you distinct. This is the line AI systems quote — no marketing adjectives.

Optional context paragraph: how the file is organized, or anything a model should know before it explores the links.

## Core Pages
- [Home](https://yourdomain.com/): What the site is and its value.
- [Product](https://yourdomain.com/product): Feature set and use cases.
- [Pricing](https://yourdomain.com/pricing): Plans and options.

## Documentation
- [Getting Started](https://yourdomain.com/docs/start): Quickstart for new users.
- [API Reference](https://yourdomain.com/docs/api): Endpoints and examples.

## Optional
- [Blog](https://yourdomain.com/blog): Articles and updates.
- [Changelog](https://yourdomain.com/changelog): Release history.

Before publishing, confirm it resolves and is served as plain text: run curl -I https://yourdomain.com/llms.txt and check the status and Content-Type, then run a link checker so every URL returns 200. Dead links erode the file's usefulness fast. If you would rather scaffold a spec-correct first draft from your existing pages, our team ships an llms.txt generator among its developer tools — but always review the output, because the curation is the part a tool cannot judge for you.

Does llms.txt actually work? The 2026 evidence

This is where most guides oversell, so here is the measured version. Adoption is real but concentrated: a study by SE Ranking across roughly 300,000 domains put adoption near 10%, heavily skewed toward technology and developer-tools sites. The recognizable adopters — Anthropic, Cloudflare, Vercel, Stripe, Mintlify, Perplexity, Cursor — are almost all in that world.

Three findings keep the hype in check. First, no major AI provider — OpenAI, Google, Anthropic, or Meta — has publicly committed to consistently fetching llms.txt as of 2026; consumption is opportunistic, not guaranteed. Second, an analysis of over 515 million LLM-bot traffic events found that the share of requests from GPTBot, ClaudeBot, and PerplexityBot actually touching /llms.txt is statistically negligible — the big crawlers are reading your normal pages, not your hint file. Third, when SE Ranking modeled whether the presence of llms.txt correlated with AI citation frequency, removing the llms.txt variable actually improved the model's accuracy. On that data, the file added noise, not signal.

So where is the genuine value? Two places, both narrow. Agent and tool retrieval: assistants like Cursor, GitHub Copilot, and Claude fetch external docs in real time, and a clean llms.txt (plus llms-full.txt) makes that retrieval cheaper and more accurate — which is why it pays off for dev-tools and documentation, where AI assistants are a real distribution channel. And agent-to-agent interoperability: Google included an llms.txt file in its Agent-to-Agent (A2A) protocol, the use case with the strongest documented evidence today.

The honest bottom line for mid-2026: if you are doing AEO, SEO, or GEO, you do not strictly need an llms.txt file. It is a recommendation, not a standard, and skipping it costs you nothing in rankings or citations. Ship it when the cost is low and agents already consume your content — documentation, developer tools, agent-facing commerce. Otherwise, put the same effort into the structural content work that genuinely drives AI citations: answer-first pages, clean structured data, and real authority. If you want the agent-readiness layer built and validated properly — llms.txt and llms-full.txt, Markdown content negotiation, AI-crawler rules, and a machine-readable API catalog, all wired into a real AEO/GEO strategy — that is exactly what our team does as a SEO, AEO & GEO service with agent-readiness.

FAQ

Frequently asked questions

Is llms.txt an official standard?

No. It is a community convention proposed by Jeremy Howard of Answer.AI in September 2024, with the spec at llmstxt.org. As of 2026 it has not been ratified by the IETF, W3C, or any standards body, and there is no central authority enforcing it.

Does llms.txt help SEO or AI rankings?

There is no evidence that it does. An SE Ranking study across ~300,000 domains found that the presence of llms.txt did not correlate with how often a site was cited by AI — removing it from their model even improved accuracy. Treat llms.txt as a retrieval aid for agents, not a ranking or citation lever.

Where do I put the llms.txt file?

At the root of your domain, so it resolves at https://yourdomain.com/llms.txt. Serve it as plain text with UTF-8 encoding, use only absolute https:// URLs inside it, and make sure every link returns a 200 status.

What is the difference between llms.txt and llms-full.txt?

llms.txt is a curated index — a short Markdown map of your most important pages. llms-full.txt is a companion file that concatenates the full Markdown body of those pages into one file, separated by --- dividers, so an agent can load your whole relevant corpus in a single fetch.

Do OpenAI, Google, or Anthropic read llms.txt?

None of them has publicly committed to consistently fetching llms.txt as of 2026, so consumption is opportunistic rather than guaranteed. The best-documented uses today are real-time documentation retrieval by coding assistants and Google's inclusion of llms.txt in its Agent-to-Agent (A2A) protocol.

How many links should an llms.txt file have?

Keep it curated — the practical 2026 guidance is roughly 3–5 H2 sections with 10–20 high-value links total, each with a short description of what the page answers. It is a table of contents, not your full sitemap; dumping every URL is the most common mistake.

Stay visible to AI

AEO, GEO, and agent-readiness tips, sent straight to your inbox.