Concepts13 min readUpdated October 24, 2026

Methodology

How mttr reads a site, scores its pages, ranks it against other sites, places it in a line of work and looks for it in an assistant’s answers. Every figure here comes from the code that does the work.

This page as Markdown

How does mttr reach a score?

It reads up to 100 public pages of a site under its own name, checks every page against one rulebook of 44 rules in four stages, and adds the stages by their share of 100 points. Code decides every check. Where a model is asked and is unsure, the page waits for a person. A score says pages are well formed. It does not say an assistant cites the site.

Step	What happens	Decided by
Read	Up to 100 public pages, 6 at a time, `robots.txt` obeyed	Code
Check	44 rules, each a check on the site or on every page it applies to	Code, and a model for 2 rules
Score	Each stage the weighted share of its checks that pass, the four added by their share	Code
Rank	A place among the 257,624 ranked sites in the US and Canada	Code
Place	A line of work, from what the site’s pages mean	Code, on a model’s vectors
Cite	Whether an assistant’s answer lists the site among its sources	Code, from the sources

How is a site read?

Only by its public address, with every request named mttr/0.1 (+https://mttr.si). mttr reads the HTML each page’s server sends, as a crawler does first, and does not run the page’s scripts, so words written in later by script are not seen. The docs on reads give the same steps in short.

The address
The domain must resolve to a public address. Every fetch checks it again, redirects too, so a name that points inside a network is never read. Only http and https on ports 80 and 443 are fetched, never an address with a sign-in in it.
robots.txt
It is fetched before any page and obeyed path by path, as RFC 9309 says: the rules written for mttr when there are any, else the rules for every crawler. The longest matching rule decides, Allow wins a tie, and a path no rule matches may be read. A path is asked with and without its last slash, and kept out if either is.
The sitemap and llms.txt
Every sitemap robots.txt names is read, and /sitemap.xml as well. /llms.txt is fetched to see whether there is one.
The home page
It is read first and by itself, since the rules treat it as the home page. It is always read, so a report can say when a site is closed.
The pages
Then the sitemap’s pages in its order, then the pages they link to, 6 at a time. When one comes back the next goes out, so a slow page holds up only itself. The read stops at 100 pages.
The four forms of the address
While the pages are read, http, https, www and the bare name are each asked for once, to see that all four end at one address.

Limit	Value
Pages in a read	100
Pages of one site fetched at once	6
Time a page has to answer	20 seconds
Redirects followed	5
Reads of one site	One every 10 minutes, whoever asks
A site closed to every crawler	The home page alone

What a read skips

What	What happens
A path `robots.txt` keeps mttr from	Never fetched and never counted. A redirect that lands on one is dropped and its place given back
Images, PDFs, stylesheets, scripts, data, archives, media, fonts and office files	Never fetched, known by the end of the address
An answer that is not HTML, such as Markdown or plain text	Not graded and its links not followed. It still takes a place in the read
One page at two addresses, by a last slash or a redirect	Read once, and the second place given back
A page that fails to load	Counted once, by the rule that pages load, and by no other rule
A page behind a sign-in, a form, or an address inside a network	Never read

Never more than was read

A read that stops at 100 pages cannot tell a page no link leads to from one it did not reach, so it reports neither orphans nor gaps in what the site covers. The report says it stopped. See Stopped at the limit.

What does the rulebook check?

44 rules in four stages, in the order a crawler meets a site. 13 are about the site as a whole, 29 are checked on each page they apply to, and 2 are read by a model. Each rule has a weight from 1 to 5 and a fix. The rulebook lists every one, and this page does not restate them.

Stage	What it asks	Rules	Share of the score
Access	Can it be fetched and indexed at all?	16	35%
Understand	Can a machine tell what each page says, and lift an answer from it?	17	30%
Trust	Is there a reason to believe it, and to cite it?	9	20%
Act	Can the visitor, or an agent, take the next step?	2	15%
All		44	100%

How a stage scores

A rule about the whole site is one check. A rule about pages is one check on every page it applies to, so a rule on 100 pages is 100 checks. A stage is the share of its checks that pass, each counted by its rule’s weight, rounded to a whole number. A heavy rule failing on many pages costs the most.

A check that waits for a person is left out until a person reads it, so it neither raises nor lowers a stage.
A rule that throws on a page holds that page for a person, and the read goes on.
A stage with no checks counted, such as one whose rules apply to no page read, counts as 100.

How the stages add up

Each stage is multiplied by its share, the four are added, and the sum is rounded. For a site whose stages came out like this:

Stage	Stage score	Its share	Adds
Access	92	35%	32.2
Understand	75	30%	22.5
Trust	60	20%	12.0
Act	100	15%	15.0
Score			82

When a site is closed

3 rules can close a site: let crawlers in, fix pages that fail to load and allow pages to be indexed. One closes it when it fails for the whole site, on the home page, or on more than half the pages read. A closed site’s score is held at 20 or less, whatever else passes, since a score for pages nobody can reach would mislead. One broken link or one page kept out of search on purpose is a fix, not a closed site.

Rules that apply to some sites

By the kind of page. Each page is taken as the home page, an article, a service or another page, from its address and its structured data. 13 rules ask only of some kinds, and 5 only of articles, such as showing when a guide was updated. Kinds are guessed, and a service page at an address that does not say so can be asked fewer rules than it should.
By what the site is. 1 rule asks only of some sites. A tap-to-call link is asked only of a site that serves a place: one that writes out a phone number or a street address, or gives a phone, an address or a local business in its structured data.
By whether it is live. A draft read sets aside the 3 rules a site not yet open cannot pass, such as a sitemap search engines can see, so its score says how ready it is for the day it opens.

Rules a model reads

2 rules need reading, not counting, such as whether a page’s first paragraph answers the question above it. A model reads those pages and gives a verdict with how sure it is. Code then decides what the verdict is worth.

The model’s reply	What mttr does
In the form asked for, finished, and at least 70% sure	Counts it: the page passes or fails the rule
Not in the form asked for, or cut off	Holds the page for a person
Less than 70% sure	Holds the page for a person, with what the model said

A model gives opinions, code decides

A model’s answer never passes or fails a page on its own, and a page it was unsure of counts for nothing until a person reads it. See Needs a person.

How fixes are ordered

Each failing rule is a fix. Fixes are ordered by the rule’s weight times the share of its checks that fail, so a heavy rule failing on a few pages can sit below a light one failing on most. A rule that is closing the site always comes first. Each fix shows the points the score would gain if every page passed it, worked out from the same counts the score is.

How are sites ranked?

By score, among the latest read of every ranked site. 257,624 sites are ranked today. A site is ranked when its latest read is neither a draft nor closed to crawlers, and its name is in the market mttr ranks: the United States and Canada. A site outside it keeps its report and is on no board.

A name ending in	Ranked
`.us` or `.ca`	Yes
A name used everywhere, such as `.com`, `.org`, `.net` or `.io`	Yes
Another country’s, such as `.uk` or `.de`	No
A region elsewhere, such as `.eu` or `.london`	No

Ties

No two sites share a place. Sites on the same score are placed by their standing: how far past the score they go, from how fast and lean their pages are, how much they say and how much of it is their own, how few of their pages show signs of generated text, whether they let AI crawlers in, how they are improving, and how long the domain has been held. Each measure is scored on a fixed curve, never against other sites, so a standing moves only when its own site changes. Its weights are ours and are not published. Past the standing, the site that reached the score first goes ahead, then the name.

Tiers

A tier is where a site’s place falls among every ranked site. First, second and third each have a mark of their own.

Tier	Takes
Top 0.1%	A place in the first 0.1% of ranked sites
Top 1%	A place in the first 1% of ranked sites
Top 5%	A place in the first 5% of ranked sites
Top 10%	A place in the first 10% of ranked sites
Top 25%	A place in the first 25% of ranked sites
Top 50%	A place in the first 50% of ranked sites
Beats a share	Below the top half, the share of ranked sites that score less

Boards

The leaderboard ranks every site, and has a board for each line of work and each state once 20 or more of its sites are ranked. A site’s place on a smaller board is counted the same way, among that board’s sites only. A place in the past counts only the sites read by then.

How does mttr tell what a site is about?

From its own words. Each page’s main text is cut into passages of about 120 words, each ending at a sentence, and an open model running on our own machines turns each passage into a vector of its meaning. A page gives at most its first 24 passages, about 2,900 words, which is the whole of nearly any page. Text past that is not searched by meaning.

A page’s meaning is the centre of its passages, and a site’s meaning is the centre of its pages.
Sites and pages that read alike are close. Close means similar, never better, and never cited.
A page’s text is not sent to an outside service to be understood.

Lines of work

There are 16 lines of work, each a short description of the work, from trades and home services to cities and public services. Why mttr shows each one with a question an assistant is asked in it. A site is placed in the one its meaning reads closest to, only when it is at least 0.45 close and 0.015 closer than the next. A site close to none is placed in none. It is an estimate: a site that does two kinds of work is placed by the one it says most about.

How does mttr tell whether an assistant cites a site?

It asks an assistant the questions people ask, and reads the sources the answer gives. Today that is Gemini, with Google Search on. Only the question is sent: nothing about the site, the person who asked, or mttr. Code, not a model, decides from the sources whether a site was cited.

The questions

Part of the bank	What is asked
Questions in the bank	2,142
Local lines of work	9, each asked about 38 cities
Cities	15 large cities across the country, and 23 in the Midwest
Lines asked without a place	7, and search and AI visibility, mttr’s own subject
Smaller towns	Trades questions are asked about towns too, the towns claimed sites serve first
How often	Each question at most once a week

Asking costs money every time, so it runs within a daily and a monthly budget, checked before every ask. When the budget is spent, or asking is paused for cost, nothing is asked. Answers are refreshed when they are asked again, and the answers a report shows are the latest kept.

Deciding a citation

A site is cited when one of the answer’s sources is its domain or a name under it. Its place is where its first source stands among them.
A site named in the answer’s words but missing from its sources is a mention, kept apart, and not a citation.
A report shows only answers to questions about what its site does: a question’s meaning must read 0.30 close to the site’s or closer, and a question that names a kind of business needs the site to be that kind, or 0.50 close when its kind is not known. With nothing that fits, there is no section.

What the answers show

Measure	Result
Sites Gemini answered questions about, set against their scores	1,890
How often a cited site outscored an uncited one	85%

That is a correlation. It does not show that a well formed site is cited because it is well formed, and a site may be cited for being large, known or local, which no rule measures. Each week a model is fit on what cited sites have in common, tested on questions it never saw. Until it orders sites better than the score alone does, nothing it finds moves a weight or a rule.

A site’s own searches

For a site whose owner connects Google Search Console, mttr asks the searches people really use to find it: its 10 seen most, two words or more, none naming the site. Each is asked at most once a week across sites, with only the question sent. As the score docs say, 7 of the sites run with mttr are cited for their own searches. Results are given as counts, and no such site is named.

Well formed is not the same as cited

A score says pages are well formed. It does not say an assistant cites the site. An assistant answers differently from one day, place and person to the next, so an answer is one sample, not a ranking.

What mttr does not claim

That a score brings rank, traffic, citations or sales. It says how well formed the pages read are.
More than was read. A read that stops at its limit says so, and every figure in it is for the pages it reached.
That a guess is a fact. A model’s unsure verdict waits for a person, and a line of work is an estimate.
That one assistant speaks for all. Answers are Gemini’s, on these questions, in these weeks.
That a figure is real when it is not. A sample on a page is marked as a sample, and a part that is not built says so.
That a score holds still. The rulebook changes, and History says when two reads were judged by different rulebooks.

What does mttr keep, and how do I keep it out?

mttr keeps what it reads so a later read can be set against an earlier one, and names sites on its public pages. A site’s owner can have it go by a code, keep mttr out, or have it removed. The privacy policy says the rest.

What	Kept
The score, the stages, and each rule with how many pages fail it	Every read
Each page’s status, depth, words, links and the rules it fails	Every read
Each page’s text, and its passages’ vectors	Once a version of the text
Each page’s HTML	The newest only, so a changed rulebook can score it again without fetching the site
The assistants’ answers and their sources	Each time a question is asked

Sites read are named on the leaderboard and in their reports. A site read from its owner’s Search Console, and any whose owner asks, goes by a random eight-letter code instead. The code’s page shows its score, place and stages, never its name.
To keep mttr out, add User-agent: mttr and Disallow: / to robots.txt. It is obeyed on the next read. See Keeping mttr out.
To have a site removed, or named by a code, ask through the form on the page for teams. A removed site is not read again.