Docs
Every technology that makes a website legible to AI agents: what each one is, how widely it is really adopted, what Lumar checks when a site publishes it, and a correct and a broken example of each.
Can an agent find the site's pages at all? Read the category.
Canonical URL web baseline ► flat
Tells an agent which address is the real one for this page.
HTTP Link header web baseline ► flat
Relations declared in headers, readable without parsing the body.
XML sitemap web baseline ► flat
The URL inventory agents and crawlers use for discovery.
llms.txt early production ► flat
A hand-written index of a site's most useful pages, written for language models.
Once an agent has a page, can it read it cheaply and correctly? Read the category.
Server-side rendering web baseline ► flat
Whether the content exists before JavaScript runs. Most agents never run it.
Markdown twin early production ▲ rising
A clean low-token markdown version of the page for agents.
JSON-LD web baseline ► flat
Structured data describing what this page is about.
Language and encoding web baseline ► flat
Declared language and charset, so text is interpreted correctly.
What has the site said agents may do with its content? Read the category.
robots.txt web baseline ► flat
The crawl-policy file every agent reads first.
AI-bot rules platform default ▲ rising
Named rules for GPTBot, ClaudeBot and friends, rather than one blanket policy.
Content Signals platform default ▲ rising
Cloudflare's robots.txt vocabulary for search, AI input and AI training.
ai.txt specification only ► flat
Attribution preferences for AI systems that use the content.
TDMRep early production ▲ rising
The W3C text and data mining reservation, used mainly by publishers.
AIPREF Content-Usage specification only ► flat
The IETF's standards-track vocabulary for stating how content may be used by AI systems.
Can an agent do something here, not just read? Read the category.
WebMCP early pilots ▲ rising
In-page tools a browsing agent can call directly.
MCP server card early pilots ▲ rising
Advertises a Model Context Protocol endpoint agents can call.
MCP authorization early production ▲ rising
The OAuth discovery chain protecting that MCP endpoint.
OpenAPI description web baseline ► flat
A machine-readable contract for the site's API.
NLWeb endpoint early pilots ► flat
Microsoft's conversational /ask route returning schema.org answers.
schema:Action web baseline ► flat
Declares actions an agent could take from this page.
auth.md early pilots ▲ rising
Telling an agent how to register for an account and get its own credentials, without a signup form or a consent screen.
Can an agent understand and act on what the site sells? Read the category.
UCP profile platform default ▲ rising
Universal Commerce Protocol discovery for agentic shopping.
Product / Offer schema web baseline ► flat
Price, availability and identifiers an agent can quote.
ACP early pilots ▼ receding
The Agentic Commerce Protocol, which powered checkout inside ChatGPT.
AP2 early pilots ► flat
The Agent Payments Protocol, covering how an agent proves it was authorised to pay.
x402 early production ▲ rising
Paying for a request with the HTTP 402 status code, so an agent can settle a charge inline.
MPP (Machine Payments Protocol) early pilots ▲ rising
Publishing an API's prices in its OpenAPI document, so an agent knows what a paid endpoint costs before it calls one.
Can each side tell who the other is? Read the category.