Why our scanner shows you less red

How this was measured. On 3 October 2026 we ran Cloudflare's free checker (isitagentready.com), Vercel's Is Agentic (is-agentic.com, run by Ora) and our own scanner against a test site we built to be wrong on purpose, and against 37 real sites. Every claim about the other two tools is what they returned or published that day.

A wall of red is not a diagnosis

Run a popular agent-readiness checker on an ordinary website and you will usually get a long list of failures. Read the list closely and most of it says the same thing: the site does not publish a file.

That is not the same as a broken file. A shoe shop does not need a payment protocol for AI agents, a news site does not need an API description, and a brochure site does not need a server for agents to call. Marking all of those as failures makes a healthy site look sick, and a site owner who sees a screen of red does one of two things: ignores it, or starts adding files nobody asked for.

Our scanner works the other way round. It lists what a site has chosen to publish, checks each of those against the rules that apply to it, and says nothing at all about what the site has not adopted.

We built a site that is wrong on purpose

To compare checkers fairly you need a site where the right answer is known. Ours serves every technology twice: correctly on one address, and published but wrong on another. The broken address has 12 technologies with mistakes planted in them, from a robots.txt with missing colons to a sitemap of relative links.

One of the broken files publishes its private signing key, so anyone who reads it can sign requests as that site's bots. Cloudflare's checker marked it as a valid key directory.

A fair caveat: the test site is ours, and its mistakes are the ones we chose to check for. But a leaked private key, a robots.txt line without a colon and a sitemap of relative links are wrong under anyone's reading of the rules, and a checker that cannot tell the correct site from the broken one is not measuring correctness.

Then we tried real sites

On the 37 real sites the pattern held. Cloudflare's checker reported broken agent cards and MCP server cards on sites that publish neither: what it had fetched were a newspaper's paywall page, a login page and a storage error. It passed a shop that sends its whole web page labelled as Markdown, and it passed the API catalog on isitagentready.com itself, which is served without the content type the standard requires.

Is Agentic is the more careful of the two, and its methodology says that "not-applicable checks are excluded". In practice it failed the BBC, The Guardian and the New York Times on an essential check for not publishing an API description, and failed 19 sites on an essential check for not serving Markdown at all.

The web is in a transition, and that calls for explanation

Most of these standards are months old. Some will matter a great deal. Many will be gone in a year, and nobody yet knows which.

llms.txt is the clearest example. In our survey of 150,074 domains, 13,730 publish one and 1,295 of those break the format. Meanwhile server logs reported by Ahrefs found that 97% of llms.txt files received no AI requests at all in May 2026. A file that is broken and unread costs a team twice: once to write it, and again whenever someone has to work out why it is there.

Some checks also point at products. Cloudflare's checker lists a failure when a site does not answer Markdown requests, and Cloudflare's own way to do that, Markdown for Agents, is available on its Pro plans and above. That does not make the check wrong. It does mean a red result is also a sales lead, and is worth reading with that in mind.

A checklist that fails you for every missing standard pushes you to adopt all of them, whether or not anything reads them. We would rather tell you what is broken in what you chose to publish, explain the rest, and let you decide.

How we decide what to flag

• Only what you published and got wrong. A technology you have not adopted produces no finding, ever.
• Every finding names its source. Of our 168 checks, 64 break a rule a specification makes mandatory. The rest say plainly whose advice they are: a vendor's, our own, or a pattern we measured.
• "Could not check" is not "missing". When a site blocks us or a request times out, we say so, instead of guessing.
• Every verdict shows its evidence. Each finding lists the requests behind it, so you can repeat them yourself.
• New checks are measured before they ship. We run each one against real published files first. One check about cache headers fired on 11 of 14 careful sites, so it became a note rather than a fault.

What to do with this

Fix what is broken in what you already publish: those files are being read, and a broken one tells an agent something false. Then adopt a new standard when something you care about actually reads it, not because a checklist is red.

Scan your site to see only what is wrong in what you publish, with the evidence behind each finding. Cloudflare's own announcement explains what its score measures, if you want to compare for yourself.