What is llms.txt?
Definition
llms.txt is a proposed file, published at a website's root or under a sub-path, that gives language models and AI agents a short Markdown summary of the site plus links to its most useful pages. Jeremy Howard published the proposal at llmstxt.org in September 2024. It is not an official standard, and Google Search does not use it.
Also known as: llms.txt file, llms.txt proposal
A proposal, not a standard
llms.txt was proposed by Jeremy Howard of Answer.AI on 3 September 2024; the current text is version 2, revised in August 2026. It has not gone through IETF, W3C or any other standards body, and its definition lives on llmstxt.org. The reasoning behind it: HTML pages bury information among navigation, ads and scripts, converting them back into clean text is imprecise, and a context window is too small for most websites in full. A short, curated guide in one predictable place is meant to give agents what they need.
What the file looks like
The file is Markdown and can live at /llms.txt or under a path such as /docs/llms.txt, in which case it covers only the pages below that path. Sections follow a fixed order. The only required one is an H1 with the site or project name; it is followed by a blockquote summary, optional paragraphs without headings, and H2 sections containing lists of links. A section titled "Optional" marks secondary links an agent can skip when context is tight.
# Example Boiler Service
> Authorised boiler servicing and repairs in Izmir, Turkey.
Prices include VAT. Weekend call-outs cover emergencies only.
## Services
- [Annual servicing](https://example.com/servicing.md): scope and duration
- [Repairs](https://example.com/repairs.md): common fault codes
## Optional
- [Company details](https://example.com/about.md)Version 2 also proposes serving a clean Markdown copy of each page at the same URL with .md appended, advertised with rel="alternate", and pointing to the covering llms.txt with rel="describedby".
Not robots.txt, not a sitemap
Despite the name, llms.txt is not an access rule. robots.txt tells crawlers where they may go; llms.txt neither blocks nor permits anything. An XML sitemap lists every indexable URL, whereas llms.txt is a short, hand-picked route through the content. The proposal's author also says it was designed mainly for inference, when an agent needs information while helping a user, rather than for model training.
Who uses it, and who doesn't
- Google Search ignores it. Google's guide to generative AI features states that you do not need LLMS.txt or similar files to appear in Search, including its AI features, that Google Search does not use them, and that having one will neither help nor harm your visibility. It also says publishing one for other services is fine.
- Software documentation uses it widely. Coding agents follow these files to find API references, many documentation platforms generate them automatically, and the major AI labs publish llms.txt files for their own developer docs. Publishing a file is not the same as an AI search product reading yours when choosing sources.
- Lighthouse checks for it, experimentally. The experimental Agentic Browsing category in Chrome's Lighthouse includes the presence of an llms.txt at the domain root among its signals.
None of the major AI search providers' crawler documentation says llms.txt is used to select or cite sources. Claims that an llms.txt file will get a site quoted by AI assistants have no documented basis.
Is it worth publishing?
For sites with technical documentation, an API or structured reference material that agents consult, it is a cheap and sensible addition. For a typical business site, the priority is that the real pages are crawlable, text-based and machine-readable; llms.txt does not substitute for that. If you do publish one, keep it current, because a summary with outdated prices and dead links misleads more than a missing file. The GEO Checker reports whether the file exists for information only; not having one is never counted as a gap.