agentic‑readiness docs
/
Scan at scale Blog Build one › Run a scan ›
Menu

Scan at scalePages with missing or invalid language/encoding declarations

Pages with missing or invalid language/encoding declarations

A per-URL report: one row per page that matches, over the pages a crawl reached.

What this report returns

The lang attribute and character encoding are how agents know what language a page is in and how to decode its bytes; get them wrong and text is mis-detected or garbled.

A row appears here when any of these is wrong, worst first:

  • Charset MissingNeither the HTTP Content-Type header nor an in-page <meta charset> declares the encoding, so agents may decode the bytes wrongly and garble the text.Fix Declare UTF-8, ideally in the header: Content-Type: text/html; charset=utf-8.
  • Lang Missing<html> has no lang attribute, so agents and translators must guess the page language and can guess wrong.Fix Declare it on the root element, e.g. <html lang="en">.
  • Lang Invalid BCP 47The lang value isn't a usable BCP 47 tag (a name like "english", or an underscore form like "en_US"), so consumers ignore it.Fix Use a valid tag such as "en" or "en-US" (hyphen, not underscore).
  • Charset Non-UTF-8The page's effective encoding isn't UTF-8, which HTML requires of every document and which modern agents assume, so characters can arrive corrupted in anything that reads it.Fix Re-save the page as UTF-8 and declare it, either as Content-Type: text/html; charset=utf-8 or as <meta charset="utf-8">: the Charset Declaration Source column says which one to change.
  • Charset LateThe charset <meta> ends after the first 1024 bytes of the document, past where parsers pre-scan, so it may be ignored.Fix Put <meta charset="utf-8"> first in <head>, or declare the charset in the Content-Type header.

The Conformance Issue Detected column is true for the three encoding rules HTML states as musts: a missing character-encoding declaration, a meta charset outside the first 1024 bytes, and a document not encoded in UTF-8. A missing lang attribute is only encouraged, and an invalid BCP 47 tag is Lumar's readiness bar, so those rows are usability problems without a spec violation.

The columns it shows

Every defect this technology can carry is a column in the grid, worst first, so nothing is hidden behind a filter you have to know to apply.

  • Language & Encoding ValidlangEncodingValid
  • Language & Encoding Conformance Issue DetectedlangEncodingConformanceIssueDetected
  • Language & Encoding FindingslangEncodingFindings
  • Lang Declared ValuelangDeclaredValue
  • Charset Declared ValuecharsetDeclaredValue
  • Charset Declaration SourcecharsetDeclarationSource
  • Charset Declaration Position (Bytes)charsetMetaByteOffset
  • Charset MissingcharsetMissing
  • Lang MissinglangMissing
  • Lang Invalid BCP 47langInvalidBcp47
  • Charset Non-UTF-8charsetNonUtf8
  • Charset LatecharsetLate

About Language and encoding

Declared language and charset, so text is interpreted correctly.

Read how Language and encoding is checked, including the specification it is validated against and a worked correct and broken example.

Running it

This report comes with Agentic Readiness in Lumar and appears on every crawl of any project that has it turned on. See the whole set, or check a single URL with the scanner on this site, which runs the same code with no account.

Get Lumar pricing