The cell nobody had decided

Two days ago a teammate testing our app with two phones in hand found a hole: person A had blocked person B, then joined a different event under a second identity, and B was visible again. The reflex is to fix that path. We had been doing exactly that for a while — test on two devices, find a wrong thing, fix the wrong thing, test again.

Then a founder named it: that loop is training on the test set. After enough rounds the system is perfect on the handful of paths someone happened to walk, and knows nothing about the rest. His rule for what two-device testing is for: ninety per cent of what it catches should be things that are impossible to catch on one device. Anything else means the thinking hadn’t been done yet.

So instead of fixing the path, we wrote down the units. Identity, account, event, surface — what each one is, and six properties that must hold no matter which combination you are in (“blocking follows the account, never the alias” is one). Then we generated the cross product: viewer identity × viewed identity × five surfaces × two context rounds. Thirty-two cells, one query, run against the real database rather than fixtures.

The first run returned three kinds of result, and only one of them was a bug.

Twenty cells said the blocking rule already held — server-side, it had been correct the whole time. The leak the two-phone session found was real but narrow, and lived in the client. That is a genuinely useful outcome: an afternoon of work we now know not to do.

One cell was a real defect, and it was one nobody would have written a ticket for. When someone switches to a second identity, the encounters they had already left behind under their first one get retroactively relabelled — the person you met last week silently becomes someone else in your history. Two functions disagreed about what an encounter is. No feature test would catch it, because it isn’t a feature; it’s a seam.

The third kind is the one I keep thinking about. One cell had no expected value at all. Not a failure — a question the product had never been asked, because no human walking through the app in a normal order ever reaches that combination. A test can only fail against a decision.When the matrix reaches a cell nobody ever ruled on, what it produces isn’t red, it’s a question.

We went on to put twenty such questions up as plain scenarios, and they were all answered in an afternoon. That reframed what the matrix is: not primarily a test harness, but an instrument that finds the parts of your own spec that don’t exist yet. The bugs were a side effect.

One caution, learned the same day: a red cell is a hypothesis, not a finding. We re-checked each one with an independent probe, and one of them turned out to be the test being wrong rather than the code. A matrix that flatters itself is worse than no matrix — it produces confident nonsense at scale.