DocsDiscoverabilityllms.txt
llms.txt
A hand-written index of a site's most useful pages, written for language models.
Why an agent cares
The idea is sound: point an agent at the pages that matter instead of making it crawl everything. Nothing consumes it, so publishing one currently changes nothing an agent does.
Why we do not score it
Adoption
Early production. Several independent organisations run it in production, though it is still niche on the open web. For this one in particular, widely published, and almost never fetched by an AI crawler.
Direction of travel: ► flat, as at .
Files are up 8.8x year on year to roughly 36,000 in a 300,000-domain sample, but Ahrefs found 97% of 137,000 files never receive an AI-bot request, and Google has said it does not and will not support it.
| Adopter | Depth | Evidence |
|---|---|---|
| Mintlify (auto-generates for all hosted docs) | default-on | mintlify.com |
| Cursor, Windsurf (docs-lookup consumers) | production | organikpi.com |
Specifications
| Document | Revision | Kind |
|---|---|---|
| llms.txt proposal | living | specification |
Last re-read against the published documents: .