Language and encoding — agent readiness

Why an agent cares

An undeclared or late charset makes an agent guess, and a wrong guess corrupts every non-ASCII character it reads. A malformed lang tag is a smaller problem but the same kind: the declaration is there and cannot be used.

Adoption

Web baseline. Standard equipment on the web. Its absence is the exception rather than its presence.

Direction of travel: ► flat, as at 2026-07-28.

BCP 47 and UTF-8 are near-universal; UTF-8 is on roughly 98% of pages (W3Techs).

What we check

Absence is never a finding. Every check below runs only once langEncodingPresent is true, so a site that has not adopted this is not assessed on it and appears in no report. 5 of these 5 can fail it. In a Lumar crawl these land on the page's own row.

Check	Severity	Raises	On whose authority
`charsetMissing`	high	conformance, usability	specification (WHATWG HTML (Living Standard, 2026-07-28))
`langMissing`	medium	usability	specification (WHATWG HTML (Living Standard, 2026-07-28))
`langInvalidBcp47`	medium	usability	Lumar readiness profile
`charsetNonUtf8`	medium	usability	specification (WHATWG Encoding + HTML (Living Standard, 2026-07-28))
`charsetLate`	medium	conformance, usability	specification (WHATWG HTML (Living Standard, 2026-07-28))

Examples

Both of these are fixtures the test suite runs through the real collector, not snippets written for a document: the first is a shape we score as valid, the second is one we flag.

correcthtmltest/e2e/test-pages/content/correct.html

<!-- langEncodingPresent + correct: valid BCP 47 lang on <html> -->
<html lang="en">

<!-- langEncodingPresent + correct: charset first, within first 1024 bytes -->
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>How agent-readiness works — Example</title>
    <link rel="canonical" href="https://example.com/content/correct.html" />

present but wronghtmltest/e2e/test-pages/content/issues.html

<!-- langEncodingPresent: yes (a lang attribute is declared).
     langInvalidBcp47: "english" is a language name, not a well-formed BCP 47 tag (should be "en").
     (Note: "eng" would NOT be flagged — it's a valid ISO 639-3 code, just not the preferred "en".) -->
<html lang="english">
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>How agent-readiness works — Example</title>

<!-- charsetLate + charsetNonUtf8: charset is declared after a large block (past the
         first 1024 bytes) and as a non-UTF-8 legacy value (ISO-8859-1). -->
    <meta name="description" content="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" />
    <meta charset="ISO-8859-1" />

Specifications

Document	Revision	Kind
BCP 47 — Tags for Identifying Languages	`BCP 47`	specification
WHATWG HTML — the lang attribute	`Living Standard`	specification
WHATWG Encoding Standard	`Living Standard`	specification
WHATWG HTML — meta charset (within the first 1024 bytes)	`Living Standard`	specification

Last re-read against the published documents: 2026-08-12.