Guides
How search engines and AI assistants read a site, and what to change. Each one is counted over the sites mttr has read and ends with something to do on your own.
8 guides and articlesCounted over 316,711 sitesUpdated
Crawlers and files
Grouped by what you are trying to do. Each figure is the one the piece turns on.
Crawlers and files
Who reads the site, and what you hand themWhich AI crawlers read your site, and who blocks them?
Which crawlers exist, what robots.txt does and does not control, and how many sites block each one.
4.6%of sites block an AI crawler by name4 minGuideWhat is llms.txt, and does anyone read it?
What the file is, whether anything reads it, and how to write one worth keeping.
21%of sites serve one4 minCounted over the 316,711 sites mttr has read, each at its latest read, as of October 26, 2026, except where a row says otherwise: the livecard figure is X's own. A guide is counted again as sites are read; an article keeps the day it was counted. A score or a rule passed says pages are well formed. It does not say an assistant cites the site. Every guide is also at its own address with .md added, for an agent.
Read your own site
The rules behind every figure here, on your pages, with the fixes worth the most first. Free, about a minute.