agentic‑readiness docs
/
Scan at scale Blog Build one › Run a scan ›

Scan at scaleSites with an invalid TDM reservation (TDMRep)

Sites with an invalid TDM reservation (TDMRep)

A site-wide report: one row per host, over the access rows the crawl emitted.

What this report returns

TDMRep (W3C) is a machine-readable text-and-data-mining opt-out served at /.well-known/tdmrep.json; if it can't be parsed, TDM agents may treat the site as having no reservation at all.

These sites' file breaks the profile: not a JSON array of rules, a rule missing its location, a tdm-reservation that isn't the number 0 or 1, or a tdm-policy that isn't an absolute URL.

How to fix it. Serve an array like [{"location": "/", "tdm-reservation": 1}], with the reservation as a bare number and any policy as a full URL.

The columns it shows

Every defect this technology can carry is a column in the grid, worst first, so nothing is hidden behind a filter you have to know to apply.

  • Hosthost
  • TDMRep ValidtdmRepValid
  • TDMRep FindingstdmRepFindings
  • TDMRep Rule CounttdmRepRuleCount
  • TDMRep Invalid Policy SamplestdmRepInvalidPolicySamples
  • TDMRep Malformed DocumenttdmRepMalformedDocument
  • TDMRep Invalid ReservationtdmRepInvalidReservation
  • TDMRep Missing LocationtdmRepMissingLocation
  • TDMRep Invalid PolicytdmRepInvalidPolicy

About TDMRep

The W3C text and data mining reservation, used mainly by publishers.

Read how TDMRep is checked, including the specification it is validated against and a worked correct and broken example.

Running it

This report ships in the Lumar agent-readiness container and appears on any project it is linked to, on every crawl. See the whole set, or check a single URL with the scanner on this site, which runs the same code with no account.

Get Lumar pricing