Scan at scaleSites with a broken llms.txt
Sites with a broken llms.txt
A site-wide report: one row per host, over the llms_txt rows the crawl emitted.
What this report returns
llms.txt is a hand-written Markdown index at /llms.txt that points an agent at the pages worth reading instead of making it crawl the whole site.
These sites publish one whose structure an agent can't rely on: no H1, which is the one element the format requires, so the file never names the site; content pushed above the H1, so a reader taking the first block as the title takes the wrong one; a sub-heading inside the description block, which the format reserves for sections of any type except headings, so a parser can read it as the start of a file list;.
How to fix it. Start the file with an H1 naming the site ("# Acme Corp"), put the summary under it as a "> " blockquote, and keep headings out of the block between the summary and the first "## " file list. llms.txt Multiple H1 is informational and never fails the file: the format names one H1 as the site's name but does not forbid others.
Spec: the llms.txt proposal at llmstxt.org (v2). Sites that publish no llms.txt don't appear here, because absence is not a failure.
The columns it shows
Every defect this technology can carry is a column in the grid, worst first, so nothing is hidden behind a filter you have to know to apply.
- Host
host - llms.txt Valid
llmsTxtValid - llms.txt Findings
llmsTxtFindings - llms.txt Title
llmsTxtTitle - llms.txt Sections
llmsTxtSections - llms.txt Link Count
llmsTxtLinkCount - llms.txt Missing H1
llmsTxtMissingH1 - llms.txt Content Before H1
llmsTxtContentBeforeH1 - llms.txt Heading In Details
llmsTxtHeadingInDetails - llms.txt Multiple H1
llmsTxtMultipleH1
About llms.txt
A hand-written Markdown index of a site's most useful pages, written for language models.
Read how llms.txt is checked, including the specification it is validated against and a worked correct and broken example.
Running it
This report ships in the Lumar agent-readiness container and appears on any project it is linked to, on every crawl. See the whole set, or check a single URL with the scanner on this site, which runs the same code with no account.