llms.txt adoption: the top 100 domains run 1.5x ahead of everyone else

In our July 2026 crawl of the Tranco top 5,000 domains, 11% of the top 100 serve a spec-valid llms.txt against 7.4% for ranks 101–5,000 — a 1.5x gap. The biggest sites on the web are adopting the AI-crawler index faster than everyone below them, while overall adoption sits at 7.5%.

The numbers come from our own monthly crawl — every domain fetched directly, files parsed against the [llmstxt.org spec], full method and CC BY 4.0 dataset on the study page .

The July 2026 snapshot

A few more cuts from the same run, dated July 11, 2026:

• 375 of 5,000 domains serve a spec-valid llms.txt; another 411 serve something malformed at that path — HTML, redirects, half-formatted Markdown.
• 31.8% of domains didn't respond to a plain 5-second fetch at all. Of the 3,410 that did respond, valid files run at 11%.
• Only 1.4% ship the llms-full.txt companion.

More files are malformed than valid. Publishing is easy; following a one-page spec apparently isn't — which is why the validator exists.

What the top-100 gap suggests

The sites with the most to lose from AI answers — and the best-staffed platform teams — are moving first. That's the usual shape of an early standard: the head adopts while the middle waits for proof. Whether crawlers reward the file is still contested (our llms.txt guide referees that fight honestly), but the cost side is settled: a curated index is an afternoon of work.

If you want to be counted in the valid column, the generator builds a spec-clean file in the browser, and the crawl re-runs monthly on a cron — the trend line updates itself on the study page .