# Scan at scale The scanner on this site checks one URL at a time. In Lumar, the same checks run across an entire site on every crawl, and arrive as 39 predefined reports you can filter, trend and assign. [Get Lumar pricing →](https://www.lumar.io/pricing/) ## Every page and every host, in one crawl Everything the single-URL scanner does runs inside a normal Lumar crawl. Turn on Agentic Readiness for a project and every crawl scores agent readiness alongside the SEO, accessibility and site-speed metrics already being collected. There is nothing separate to schedule and no second crawl to pay for. The checks land in two places, because agent-facing technologies live in two places. **Page signals** vary per URL — a canonical, Product schema, an in-page tool declaration — so they attach to the page row, and you filter them like any other page metric. **Site-wide signals** are one fact per host — robots.txt, an `llms.txt`, a sitemap, an MCP server card, a live probe of the endpoint it advertises — so they are fetched once per host and arrive as their own rows rather than being copied onto every URL. That split is what makes this workable on a large site. A million-page crawl does not fetch robots.txt a million times, and a host-level fault is reported once rather than as a million identical page issues. ## It reports what is broken, never what is absent No report on this page will tell you that you have not adopted something. These are emerging technologies and most sites have not adopted most of them, which is not a finding. Every report is gated on the technology actually being present, so a page enters one only when the site tried to implement something and got it wrong. That is what makes the output a work queue rather than a scorecard. A row is a thing a team published, that does not do what it was published to do, with the specific defect named — a relative canonical, a sitemap listing URLs outside its own folder, an agent card missing a required field, an MCP endpoint that does not answer the protocol its card advertises. Where we could not check something, the row says so rather than guessing. A timeout or a blocked connection is us failing to look, and it is reported as unassessed, never as a fault on the site. ## The 39 reports Grouped the way they appear in Lumar. Each one opens a page explaining what it returns, which columns it shows, and which technology it is about. ### Overview - [Sites publishing agent-ready signals](https://agentic-readiness.lumar.io/scan-at-scale/site-wide-signals-detected)site-wide - [Sites with a misconfigured agent-ready signal](https://agentic-readiness.lumar.io/scan-at-scale/broken-site-wide-signals)site-wide ### Discoverability - [Pages with a broken canonical](https://agentic-readiness.lumar.io/scan-at-scale/broken-canonical)per-URL - [Pages with a relative canonical URL](https://agentic-readiness.lumar.io/scan-at-scale/relative-canonical)per-URL - [Pages with broken Link headers](https://agentic-readiness.lumar.io/scan-at-scale/broken-link-headers)per-URL - [Sites with a broken sitemap](https://agentic-readiness.lumar.io/scan-at-scale/broken-sitemap)site-wide - [Sites with a broken llms.txt](https://agentic-readiness.lumar.io/scan-at-scale/broken-llms-txt)site-wide ### Content - [Sites with broken Markdown negotiation](https://agentic-readiness.lumar.io/scan-at-scale/broken-markdown-negotiation)site-wide - [Pages with invalid JSON-LD](https://agentic-readiness.lumar.io/scan-at-scale/invalid-jsonld)per-URL - [Pages with unparseable JSON-LD](https://agentic-readiness.lumar.io/scan-at-scale/jsonld-unparseable)per-URL - [Pages whose content only appears after JavaScript](https://agentic-readiness.lumar.io/scan-at-scale/content-hidden-without-js)per-URL - [Pages with a broken Markdown twin](https://agentic-readiness.lumar.io/scan-at-scale/broken-markdown-twin)per-URL - [Pages with missing or invalid language/encoding declarations](https://agentic-readiness.lumar.io/scan-at-scale/invalid-lang-encoding)per-URL ### Access - [Sites with a malformed robots.txt](https://agentic-readiness.lumar.io/scan-at-scale/malformed-robots-txt)site-wide - [Sites with broken AI bot rules](https://agentic-readiness.lumar.io/scan-at-scale/broken-ai-bot-rules)site-wide - [Sites with a misspelled AI bot token](https://agentic-readiness.lumar.io/scan-at-scale/misspelled-ai-bot-token)site-wide - [Sites with invalid Content Signals](https://agentic-readiness.lumar.io/scan-at-scale/invalid-content-signals)site-wide - [Sites with an invalid ai.txt](https://agentic-readiness.lumar.io/scan-at-scale/invalid-ai-txt)site-wide - [Sites with an invalid TDM reservation (TDMRep)](https://agentic-readiness.lumar.io/scan-at-scale/invalid-tdmrep)site-wide ### Capabilities - [Pages with broken WebMCP tools](https://agentic-readiness.lumar.io/scan-at-scale/broken-webmcp)per-URL - [Pages with undescribed WebMCP tools](https://agentic-readiness.lumar.io/scan-at-scale/webmcp-undescribed-tools)per-URL - [Pages with a broken schema.org Action (potentialAction)](https://agentic-readiness.lumar.io/scan-at-scale/broken-schema-action)per-URL - [Sites with a broken MCP server card](https://agentic-readiness.lumar.io/scan-at-scale/broken-mcp-server-card)site-wide - [Sites with an invalid OpenAPI contract](https://agentic-readiness.lumar.io/scan-at-scale/invalid-openapi)site-wide - [Sites with a broken API catalog](https://agentic-readiness.lumar.io/scan-at-scale/broken-api-catalog)site-wide - [Sites with a broken Personal Agent Protocol document](https://agentic-readiness.lumar.io/scan-at-scale/broken-personal-agent-protocol)site-wide - [Sites with a broken Agentic Resource Discovery file](https://agentic-readiness.lumar.io/scan-at-scale/broken-ard)site-wide - [Sites with a broken RSL licence](https://agentic-readiness.lumar.io/scan-at-scale/broken-rsl)site-wide - [Sites with a broken NLWeb /ask endpoint](https://agentic-readiness.lumar.io/scan-at-scale/broken-nlweb-ask)site-wide - [Sites with a broken NLWeb /mcp server](https://agentic-readiness.lumar.io/scan-at-scale/broken-nlweb-mcp)site-wide - [Sites with broken MCP authorization discovery](https://agentic-readiness.lumar.io/scan-at-scale/broken-mcp-auth)site-wide - [Sites with an unsafe MCP authorization setup](https://agentic-readiness.lumar.io/scan-at-scale/mcp-auth-security-risk)site-wide ### Commerce - [Pages with broken Product schema](https://agentic-readiness.lumar.io/scan-at-scale/broken-product-schema)per-URL - [Products missing core Offer fields](https://agentic-readiness.lumar.io/scan-at-scale/product-missing-offer-fields)per-URL - [Sites with a broken UCP profile](https://agentic-readiness.lumar.io/scan-at-scale/broken-ucp-profile)site-wide ### Trust - [Pages with incomplete editorial provenance](https://agentic-readiness.lumar.io/scan-at-scale/incomplete-editorial-metadata)per-URL - [Sites with a broken Web Bot Auth directory](https://agentic-readiness.lumar.io/scan-at-scale/broken-web-bot-auth)site-wide - [Sites leaking a private key in their Web Bot Auth directory](https://agentic-readiness.lumar.io/scan-at-scale/web-bot-auth-private-key-exposed)site-wide - [Sites with a broken A2A agent card](https://agentic-readiness.lumar.io/scan-at-scale/broken-agent-card)site-wide ## Running it on your own site Agentic Readiness is available to Lumar accounts and runs on crawls you are already scheduling. If you want to see what it finds on one URL first, the scanner on this site is the same code and needs no account. You can also read [what each technology is and how it is checked](https://agentic-readiness.lumar.io/docs), or the [measurements we have published](https://agentic-readiness.lumar.io/blog/agentic-readiness-of-the-web) from running these checks across 500,000 domains. [Get Lumar pricing →](https://www.lumar.io/pricing/)