- Proposed
- Jeremy Howard (Answer.AI), September 3, 2024
- Google Search
- ignores it, no ranking effect (June 2026 clarification)
- Adoption
- ~10% of 300K domains (SE Ranking); up to 28% in Ahrefs' technical sample
- Requests
- 97% of published files got zero fetches in May 2026 (Ahrefs, 137K domains)
- AI citation lift
- no measurable effect in 2025-2026 studies
- Where it works today
- developer docs read by coding assistants and agents
llms.txt is a proposed standard: a Markdown file at your domain root that lists your most important pages so language models can find them without parsing full HTML. Jeremy Howard of Answer.AI proposed it on September 3, 2024. As of July 2026, Google ignores it, no major chatbot documents using it to pick citations, and independent studies find no citation lift. It has one honest use, and it isn't SEO.
The short version: ship one if you run developer documentation, skip it if you're chasing rankings or AI citations.
What is llms.txt, exactly?
It's a single Markdown file served at /llms.txt. The proposal defines a specific structure:
- An H1 with the site or project name (the only required part).
- A blockquote summarizing what the project is.
- Optional Markdown sections with more detail.
- H2 sections holding lists of links, each formatted as
[name](url): optional note. - An optional "Optional" section for secondary links a model can skip to save context.
It is a discovery index, a table of contents that points to your real docs. It is not the docs themselves. A companion convention, llms-full.txt, dumps the entire documentation set into one file for a single-fetch ingest. Anthropic, Stripe, Vercel, Cloudflare, and Cursor all publish one, per Mintlify's roundup of real examples.
Does llms.txt help you rank or get cited?
No, not measurably, on current evidence. Three independent findings point the same way:
- Google ignores it. Google's AI-optimization guide says you don't need "machine readable files, AI text files, markup, or Markdown to appear in Google Search," because "Google Search itself doesn't use them." A June 2026 clarification added that maintaining an llms.txt "will neither harm nor help" your rankings. We cover the full debunk in Google's GEO guidance.
- Almost nobody fetches it. Ahrefs checked 137,210 domains with traffic in May 2026. About 28% published a valid llms.txt, and 97% of those files received zero requests that month. Nothing fetched them.
- No citation correlation. SE Ranking analyzed roughly 300,000 domains (about 10% had the file) with an XGBoost model. Removing the llms.txt variable improved the model's accuracy. The file added noise, not signal, to citation prediction.
Adoption skews technical, so real-world coverage is lower than the headline numbers suggest. Ahrefs flags its own 28% as an upper bound.
Who actually reads llms.txt?
| System | Reads llms.txt? |
|---|---|
| Google Search (AI Overviews, AI Mode) | No. Google states it ignores the file |
| ChatGPT / OpenAI crawlers | Not documented as a citation input |
| Claude, Perplexity | No provider documents it as a citation signal |
| Coding assistants (Cursor, Copilot, agents) | Yes, when pointed at your docs to fetch on demand |
The pattern is clear. Consumer answer engines don't advertise using the file. The value that exists today comes from developer tooling: an AI coding assistant grounding on your API docs benefits from a clean index of Markdown pages, because it skips the navigation, ads, and scripts that waste its context window.
So should you ship one?
Ship an llms.txt if:
- You publish real documentation (API references, SaaS docs, a dev tool) that assistants and agents fetch in real time.
- You can generate it from your docs automatically, so it stays current.
Skip it, or at least don't prioritize it, if:
- You expect it to move Google rankings. Google says it won't.
- You expect it to earn chatbot citations. No study supports that, and no provider claims it.
- Someone is selling "llms.txt installation" as an AI SEO service. That's the myth Google published a clarification to stop.
The file is low-effort and low-risk. Google confirmed it neither helps nor hurts. The mistake isn't shipping one; it's treating it as a growth lever. Right now it's hygiene, not a channel.
We publish one at /llms.txt for the tools that read it, and we don't count it as generative engine optimization.
What to do instead for AI visibility
The levers that actually move AI citations are the ones models can retrieve and quote: pages that rank, state facts directly, and cite sources. Start with the LLM SEO guide for what drives citations, use AI SEO tools to measure whether you're cited, and if you're new to the discipline, answer engine optimization is the ground floor. An llms.txt file is a nice-to-have on top of that work, never a substitute for it.
- https://www.answer.ai/posts/2024-09-03-llmstxt.html
- https://github.com/answerdotai/llms-txt
- https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- https://ahrefs.com/blog/llmstxt-study/
- https://www.searchenginejournal.com/llms-txt-shows-no-clear-effect-on-ai-citations-based-on-300k-domains/561542/
- https://www.mintlify.com/blog/real-llms-txt-examples
