Scan at scalePages with missing or invalid language/encoding declarations
Pages with missing or invalid language/encoding declarations
A per-URL report: one row per page that matches, over the pages a crawl reached.
What this report returns
The lang attribute and character encoding are how agents know what language a page is in and how to decode its bytes; get them wrong and text is mis-detected or garbled.
A row appears here when any of these is wrong, worst first:
- Charset MissingNeither the HTTP
Content-Typeheader nor an in-page <meta charset> declares the encoding, so agents may decode the bytes wrongly and garble the text.Fix Declare UTF-8, ideally in the header:Content-Type: text/html; charset=utf-8. - Lang Missing<html> has no lang attribute, so agents and translators must guess the page language and can guess wrong.Fix Declare it on the root element, e.g. <html
lang="en">. - Lang Invalid BCP 47The lang value isn't a usable BCP 47 tag (a name like
"english", or an underscore form like"en_US"), so consumers ignore it.Fix Use a valid tag such as"en"or"en-US"(hyphen, not underscore). - Charset Non-UTF-8The page's effective encoding isn't UTF-8, which HTML requires of every document and which modern agents assume, so characters can arrive corrupted in anything that reads it.Fix
Re-savethe page as UTF-8 and declare it, either asContent-Type: text/html; charset=utf-8or as <metacharset="utf-8">: the Charset Declaration Source column says which one to change. - Charset LateThe charset <meta> ends after the first 1024 bytes of the document, past where parsers pre-scan, so it may be ignored.Fix Put <meta
charset="utf-8"> first in <head>, or declare the charset in theContent-Typeheader.
The Conformance Issue Detected column is true for the three encoding rules HTML states as musts: a missing character-encoding declaration, a meta charset outside the first 1024 bytes, and a document not encoded in UTF-8. A missing lang attribute is only encouraged, and an invalid BCP 47 tag is Lumar's readiness bar, so those rows are usability problems without a spec violation.
The columns it shows
Every defect this technology can carry is a column in the grid, worst first, so nothing is hidden behind a filter you have to know to apply.
- Language & Encoding Valid
langEncodingValid - Language & Encoding Conformance Issue Detected
langEncodingConformanceIssueDetected - Language & Encoding Findings
langEncodingFindings - Lang Declared Value
langDeclaredValue - Charset Declared Value
charsetDeclaredValue - Charset Declaration Source
charsetDeclarationSource - Charset Declaration Position (Bytes)
charsetMetaByteOffset - Charset Missing
charsetMissing - Lang Missing
langMissing - Lang Invalid BCP 47
langInvalidBcp47 - Charset Non-UTF-8
charsetNonUtf8 - Charset Late
charsetLate
About Language and encoding
Declared language and charset, so text is interpreted correctly.
Read how Language and encoding is checked, including the specification it is validated against and a worked correct and broken example.
Running it
This report comes with Agentic Readiness in Lumar and appears on every crawl of any project that has it turned on. See the whole set, or check a single URL with the scanner on this site, which runs the same code with no account.