agentic‑readiness docs
/
Scan at scale Blog Build one › Run a scan ›

DocsBuild

Build it, free

Answer a couple of questions and walk away with either the file itself, checked against the same rules this scanner uses on live sites, or the steps to add it to your own stack.

We generate the file

Answer a question or two and the artifact comes back built and checked.

AI-bot rules A block of robots.txt rules naming the AI crawlers you choose, each with its own access, checked against your current file so it cannot duplicate a group you already have. Build your AI crawler rules → Content Signals The Content-Signal line for your robots.txt, placed inside a group so it actually applies, with each preference set the way you choose. Build your Content Signals → ai.txt A complete ai.txt stating how you want to be credited when an assistant uses your content. Build your ai.txt → TDMRep A valid /.well-known/tdmrep.json declaring your text and data mining reservation for the whole site. Build your TDMRep file → A2A agent card A valid /.well-known/agent-card.json describing your agent, its endpoint and what it can do. Build your A2A agent card → MCP server card A valid /.well-known/mcp.json pointing agents at your MCP server, with the transport it actually speaks. Build your MCP server card →

We show you how

These live in your templates, headers or code, so the wizard reads your site and writes the steps for it.

Canonical URL Where the canonical link goes, what a correct one looks like, and the two mistakes that make it useless. Set up canonical URLs → HTTP Link header How to send discovery relations in the HTTP response, and which relations agents actually read. Set up Link headers → XML sitemap How to publish a sitemap agents can enumerate, and how to point robots.txt at it. Set up an XML sitemap → llms.txt The one thing the format requires, and the four ways a published file stops being usable to an agent. Set up an llms.txt index → robots.txt The structure a crawl-policy file needs, and the ordering mistakes that silently disable a rule. Set up robots.txt → Server-side rendering Why an agent often sees an empty page, and how to check what yours returns before any script runs. Serve content without JavaScript → Markdown twin How to serve a Markdown version of a page to the agents that ask for one, and keep HTML the default. Serve a Markdown twin → Markdown negotiation How to answer a request for Markdown with Markdown from the same address, and keep caches from mixing it up with the HTML. Serve Markdown by negotiation → JSON-LD Where structured data goes, and the shape errors that make a block unreadable to an agent. Set up JSON-LD → Language and encoding The two declarations that stop an agent guessing what language your page is in and how to decode it. Declare language and encoding → WebMCP How a page offers tools to a browsing agent at runtime, and what a well-formed tool looks like. Add WebMCP tools → MCP authorization The discovery chain an agent follows from a 401, and the two documents that have to exist for it to work. Set up MCP authorization → OpenAPI description How to serve a machine-readable contract for your API, and where agents look for it. Publish an OpenAPI description → API catalog How to list your APIs at one well-known address, in the linkset format RFC 9727 requires. Publish an API catalog → NLWeb endpoint What a conversational /ask route has to return, and the response shape agents expect. Set up an NLWeb endpoint → schema:Action How to describe an action an agent can take, rather than only the content of the page. Add schema:Action markup → UCP profile The commerce profile agents read to learn what your store supports, and the fields it must carry. Publish a UCP profile → Product / Offer schema The fields an agent needs before it can quote a price, and the ones that are most often missing. Add Product and Offer markup → Web Bot Auth directory How to publish the keys that let a verified agent prove who it is, in the format verifiers read. Publish a Web Bot Auth directory → Editorial metadata How to say who wrote a page and when, in the form an agent can actually attribute. Add editorial provenance →

Not available to build yet

We track these and do not recommend implementing them, so we will not generate one. Each says what would change that.

  • AIPREF Content-UsageThe IETF's standards-track vocabulary for stating how content may be used by AI systems.We will score it when the attach draft reaches RFC. The vocab half advancing does not move that condition: only attach defines the header and robots.txt rule a crawl could read. Note what the draft does not give us yet: it states no MUST or SHOULD governing how a site expresses a preference, so today's defect surface is syntactic only: a malformed structured-field dictionary, including uppercase keys, which fail parsing. vocab-08 sections 6.4 and 6.5 make unknown labels and non-y/n values "not errors", and attach-05 section 3.2 reads a value without a leading / as having no path, so none of those is a defect.
  • auth.mdTelling an agent how to register for an account and get its own credentials, without a signup form or a consent screen.One vendor's 0.x specification with no standards body, and it still renames things a client can see: v0.6.0 (2026-06-10) moved the email path out of identity_assertion into a new service_auth type and dropped verified_email from the assertion list. Score it once the specification reaches 1.0 or is submitted to a standards body, and agent_auth blocks appear beyond the authors' own ecosystem.
  • ACPThe Agentic Commerce Protocol, which powered checkout inside ChatGPT.Since spec 2026-04-17 a site can publish a discovery document at /.well-known/acp.json (the discovery RFC is still a proposal). A probe on 2026-10-05 found no merchant or platform serving one, so there is nothing yet to score.
  • AP2The Agent Payments Protocol, covering how an agent proves it was authorised to pay.Same reason as ACP: no published artifact means no crawl-detectable failure. Donation to the FIDO Alliance in Apr 2026 makes it more likely to matter, not more visible to a crawl.
  • x402Paying for a request with the HTTP 402 status code, so an agent can settle a charge inline.A crawl sees x402 only on a 402 response to a paid resource, which a page crawl rarely requests, and there is no per-host document to fetch. Scoring it would mean probing other people's paid endpoints.
  • MPP (Machine Payments Protocol)Publishing an API's prices in its OpenAPI document, so an agent knows what a paid endpoint costs before it calls one.The discovery extension is a draft in the specification repository, not a submitted Internet-Draft, and the draft itself makes discovery OPTIONAL because the runtime 402 challenge is always authoritative. Score it once the extension is submitted and stable and more than one organisation serves a discovery document.