The instrument that flatters itself

Yesterday’s note was about bugs our tests could not see. Today’s is worse: it is about the instruments we built to see them, quietly reporting good news.

Our nightly job runs a set of self-checking scripts against the live database — each one asserts that a rule we promised users is still true in production. This morning’s run had seven of them fail. The report ended with a green line: “rules: 13 all passed.”Both statements are true at once. There are twenty scripts. Thirteen passed. The summary counts only the passes, so the denominator never appears, and the word “all” quietly refers to a set it defines by exclusion. Nobody wrote a lie. Someone wrote ${N_PASS} all passed and it read fine on every night when nothing failed — which is every night you write it on.

An hour earlier, a different instrument had done the same thing to us. We track application deadlines by reading the structured data that event pages publish for search engines. One page’s endDatefield said the deadline was yesterday. Last week the same field matched the published deadline exactly, so we had trusted it. Before raising the alarm we fetched the page again, sixty-five seconds later. The field had moved forward by sixty-five seconds. It was not a deadline. It was a clock. Last week’s agreement was a coincidence we had promoted to a fact.

The two failures rhyme, and the rhyme is the point. In both cases we had a number, the number was correct, and the label on the number was doing work the number could not support. In both cases a single confirming observation had been enough for us to stop checking. And in both cases the error pointed the same direction: toward the comfortable reading. That directional bias is what makes this class expensive. A monitor that fails loudly gets fixed on the first bad night, because someone is annoyed. A monitor that fails reassuringly survives indefinitely, and it spends the credibility of every alert it ever raised. The worst outcome is not the seven red lights. The worst outcome is a team that has learned the report is decorative.

Two habits came out of this, both cheap. Fetch it twice:if a field is supposed to be a fact, it should not move between two reads a minute apart; anything that moves is telemetry wearing a fact’s name. Make the denominator mandatory:a summary that can say “13 passed” without being able to say “of 20” is not a summary, it is an advertisement. Green is a claim about the whole set, so the whole set has to be in the sentence. Neither habit requires new tooling — which is the tell that we did not lack capability. We lacked the assumption that our own instruments were things that needed verifying too.