agentic‑readiness docs
/
Scan at scale Blog Build one › Run a scan ›
Menu

Scan at scaleSites with a broken sitemap

Sites with a broken sitemap

A site-wide report: one row per host, over the sitemap rows the crawl emitted.

What this report returns

A sitemap lets crawlers and agents enumerate a site's URLs without following every link.

A row appears here when any of these is wrong, worst first:

  • Empty DocumentThe sitemap the site declares is served with no content in it, so it enumerates nothing an agent or crawler could follow. Distinct from malformed XML, which means a document that is there and won't parse: this one has no document at all, and the repair is different.Fix Publish the URLs at that address, or remove the Sitemap: line in robots.txt that promises them.
  • Missing NamespaceThe <urlset>/<sitemapindex> does not declare the required sitemaps.org 0.9 namespace, so strict parsers do not recognise the file as a sitemap and reject the whole thing. It covers both faults: a root with no xmlns at all, and one declaring a near-miss such as the https:// spelling of the sitemaps.org URL, which is the common case and a one-character fix.Fix Declare xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" on the root element, with http and not https.
  • Malformed XMLA sitemap will not parse, so crawlers get nothing from it: either its XML isn't well-formed, or it's a .gz that doesn't decompress. A document that is simply not a sitemap is Sitemap Wrong Document Type instead, not this.Fix Repair the file (unclosed tags, unescaped & characters, or re-generate the gzip) and re-validate it. Sitemap Findings names the offending URL.
  • Wrong Document TypeA URL declared as a sitemap (a robots.txt Sitemap: line, or a <loc> in a sitemap index) serves something that is not a sitemap in any permitted format, so an agent following the declaration gets no URLs. The file itself is often fine: a site pointing Sitemap: at its llms.txt has a valid llms.txt and an invalid declaration.Fix Point the declaration at a real sitemap, or remove it. Sitemap Findings names the URL.
  • Relative Locs<loc> entries contain relative URLs, which the sitemaps.org protocol forbids, so crawlers drop those entries.Fix Make every <loc> a full absolute URL including scheme and host.
  • Relative DeclarationA Sitemap: line in robots.txt gives a path such as /sitemap.xml instead of a full URL. Google's robots.txt rules say the value must be a fully qualified URL, so a crawler may not find the sitemap from it.Fix Write the whole address, e.g. Sitemap: https://example.com/sitemap.xml.
  • Lists Another HostThe sitemap lists URLs on a host it does not speak for, so crawlers drop those entries and the pages go undiscovered.Fix Serve a sitemap from that host, or have that host's robots.txt declare this one (cross-submission), e.g. Sitemap: https://shop.example.com/sitemap.xml.
  • Too LargeA single sitemap exceeds the protocol cap of 50,000 URLs / 50 MB, past which crawlers truncate or reject it.Fix Split it into multiple sitemaps referenced by a <sitemapindex>.

A note, not a fault: we also mention when a sitemap in a folder lists pages outside that folder, which Google reads only if the sitemap was submitted in Search Console. It never fails the site.

Spec: the sitemaps.org 0.9 protocol.

The columns it shows

Every defect this technology can carry is a column in the grid, worst first, so nothing is hidden behind a filter you have to know to apply.

  • Hosthost
  • Sitemap ValidsitemapValid
  • Sitemap FindingssitemapFindings
  • Sitemap URLs ListedsitemapUrlCount
  • Sitemap Largest File Size (Bytes)sitemapBytes
  • Sitemap Relative Entry SamplessitemapRelativeLocSamples
  • Sitemap Cross-Host Entry SamplessitemapCrossHostLocSamples
  • Sitemap Entries Outside Its Folder (Samples)sitemapOutOfDirectoryLocSamples
  • Sitemap Conformance Issue DetectedsitemapConformanceIssueDetected
  • Sitemap Empty DocumentsitemapEmptyDocument
  • Sitemap Missing NamespacesitemapMissingNamespace
  • Sitemap Malformed XMLsitemapMalformedXml
  • Sitemap Wrong Document TypesitemapWrongDocumentType
  • Sitemap Relative LocssitemapRelativeLocs
  • Sitemap Relative DeclarationsitemapRelativeDeclaration
  • Sitemap Lists Another HostsitemapCrossHostLocs
  • Sitemap Too LargesitemapTooLarge
  • Sitemap Lists Pages Outside Its FoldersitemapOutOfDirectoryLocs
  • Sitemap Partially AssessedsitemapPartiallyAssessed

About XML sitemap

A list of a site's pages, published so crawlers can find them.

Read how XML sitemap is checked, including the specification it is validated against and a worked correct and broken example.

Running it

This report comes with Agentic Readiness in Lumar and appears on every crawl of any project that has it turned on. See the whole set, or check a single URL with the scanner on this site, which runs the same code with no account.

Get Lumar pricing