# Content Signals

Cloudflare's robots.txt vocabulary for search, AI input and AI training.

**[Access](/docs/access)** · checked once per host (item type `access`) · 4 checks that can fail it

It separates three uses a blanket `Disallow` cannot: appearing in search, being used as AI input, and being used for training. Publication is enormous and consumption is nil, so treat it today as an expression of preference rather than a contract.

## Adoption

**Platform default** (L3) — published on millions of sites, and no crawler has committed to honouring it yet

A major platform ships it by default or in one click, so its footprint is measurable at web scale. Direction of travel: ▲ rising, as of 2026-07-28.

Default on 3.8M+ Cloudflare-managed robots.txt domains since Sep 24, 2025, but no crawler or LLM operator has committed to honouring it.

| Adopter | Depth | Evidence |
|---|---|---|
| Cloudflare (managed robots.txt) | default-on | [blog.cloudflare.com](https://blog.cloudflare.com/content-signals-policy/) |
| Liferay (CMS feature) | production | [liferay.dev](https://liferay.dev/b/adding-content-signal-support-to-liferay) |

## What we check

Absence is never a finding: every check below runs only once `contentSignalPresent` is true, so a site that has not
adopted this is not assessed on it and appears in no report.

| Check | Kind | Raises | On whose authority |
|---|---|---|---|
| `contentSignalOutsideGroup` | high severity | usability | Cloudflare profile |
| `contentSignalConflicting` | high severity | usability | Cloudflare profile |
| `contentSignalUnknownValue` | medium severity | usability | Cloudflare profile |
| `contentSignalEmptyKeyValue` | medium severity | usability | Cloudflare profile |

## Examples

These are the fixtures the test suite runs through the real collector, not snippets written for a document.

- Correct: `test/e2e/test-pages/access/correct.robots.txt`
- Present but wrong: `test/e2e/test-pages/access/issues.robots.txt`

## Specifications

| Document | Revision | Kind |
|---|---|---|
| [Cloudflare Content Signals Policy (managed robots.txt)](https://developers.cloudflare.com/bots/additional-configurations/managed-robots-txt/) | living document | vendor profile |
| [contentsignals.org — the policy text](https://contentsignals.org/) | living document | guidance |

Last re-read against the published documents: 2026-08-12.