Scan at scaleSites with an invalid ai.txt
Sites with an invalid ai.txt
A site-wide report: one row per host, over the access rows the crawl emitted.
What this report returns
ai.txt (the Geodocs AI Agent Content Spec) states a site's AI attribution and citation preferences at /ai.txt.
These sites publish one that tools can't parse: prose instead of Key: value directives, a Source-url that isn't an absolute URL, or an Attribution-required that isn't yes/no, so the stated preferences are never read.
How to fix it. Write the file as directives, e.g. Attribution-required: yes and Source-url: https://www.example.com/.
The columns it shows
Every defect this technology can carry is a column in the grid, worst first, so nothing is hidden behind a filter you have to know to apply.
- Host
host - ai.txt Valid
aiTxtValid - ai.txt Findings
aiTxtFindings - ai.txt Directive Keys
aiTxtDirectiveKeys - ai.txt Source URL Value
aiTxtSourceUrlValue - ai.txt Attribution Value
aiTxtAttributionValue - ai.txt Invalid Source URL
aiTxtInvalidSourceUrl - ai.txt Unknown Attribution Value
aiTxtUnknownAttributionValue
About ai.txt
Attribution preferences for AI systems that use the content.
Read how ai.txt is checked, including the specification it is validated against and a worked correct and broken example.
Running it
This report ships in the Lumar agent-readiness container and appears on any project it is linked to, on every crawl. See the whole set, or check a single URL with the scanner on this site, which runs the same code with no account.