A broken tool hides the bugs it would have caught

Our mobile app has a preview harness — a debug entry point that renders every screen in isolation with fake data, so you can flip through the whole UI in one loop without tapping through the real product. Yesterday we found out it had been crashing on launch for weeks. Nobody noticed, because nobody had run it, because it was broken.

The root cause was mundane. When we localized the app, every screen started requiring localization delegates that the harness didn’t install. On top of that, individual screens had grown new provider dependencies over time, and the harness supplied providers on demand— whichever ones the screens needed back when each variant was written. So each new dependency silently broke one more preview. The harness didn’t rot all at once; it rotted one screen at a time, invisibly.

Fixing it took an afternoon. That is not the interesting part. The interesting part is what fell out of the first clean run: flipping through twenty-five screens by eye immediately surfaced two real bugs that were live in the shipping app. One was a hardcoded English string on an onboarding card, sitting in a UI we’d otherwise fully translated. The other was a pair of avatars being clipped into ellipses instead of circles, because a positioned element had been given a left offset but no width, so the clip took the image’s native aspect ratio.

Neither bug was subtle. Both would have been caught in seconds by anyone looking at those screens. Nobody was looking, because the thing whose entire job was to make looking cheap had quietly stopped working.

That reframes what the outage actually cost. The naive accounting says a broken internal tool costs you the time to fix it — an afternoon. The real accounting is that it costs you every defect it would have caught while it was down, and you never see that bill itemized, because those defects don’t arrive labeled “the harness would have caught me.” They arrive as a user’s screenshot, or they don’t arrive at all and just quietly degrade the product.

Two things we changed. First, the mechanical fix: give a preview harness the full provider set at the root, not per-screen on demand. On-demand wiring makes the harness silently coupled to the current dependency graph of every screen, which is exactly the thing that changes every week. Supplying the superset is slightly wasteful and never breaks.

Second, the habit: a tool nobody has run in a month is not “working,” it’s unmeasured. If the harness had been in the test suite — even as a single “does it launch and render forty frames without throwing” check — the localization change would have failed loudly on the day it landed, instead of handing us a bill weeks later that we paid without ever seeing the invoice.