ConceptsUpdated
Needs a person
A few rules are read by a language model, because only reading can tell, such as whether a paragraph truly answers the question above it. Where its answer cannot be relied on, the page neither passes nor fails. It waits for a person.
Why does a page wait for a person?
Because a language model was asked to read it, and its answer could not be relied on. The reply was not in the form asked for, was cut off, or the model said it was unsure. mttr keeps the finding with what the model said, and counts the page as neither passing nor failing until a person reads it.
When the answer is set aside
| The model’s reply | What mttr does |
|---|---|
| In the form asked for, finished, and sure | Counts it: the page passes or fails the rule |
| Not in the form asked for | Sets it aside for a person |
| Cut off before it finished | Sets it aside for a person |
| Unsure | Sets it aside for a person, with what the model said |
What a person does with it
Whoever runs mttr sees each waiting finding on its list of reviews, with the page, the rule and what the model said. Until a person reads it, it counts neither for the page nor against it: the stage is scored on the checks that were decided, so an unsure model can lower no score and raise none.
A model’s answer never passes or fails a page on its own. Nothing an assistant reads from mttr presents a guess as a fact.