The impatient user

A founder spent three hours yesterday trying to make a fully automated end-to-end test pass on a physical phone: signed production build, secure photo viewer, screenshot evidence, fixture cleanup. Most of the pieces worked. The whole never did — the run kept dying at the first backend checkpoint, and after a day of narrowing, the failure still lived somewhere in the production session stack. Three hours, zero tests that could close a ticket.

So he stopped. Not paused — ruled. Full-device automation was demoted on the spot, and testing was split into three lanes: humans walk the happy paths on real hardware; automation covers what a single machine can verify locally; a thin probe checks the backend on its own. Anything involving two devices, a locked screen, hardware buttons, system interface, or a minute of waiting never enters unattended automation again.

Then he picked up the phone and did the thing the robots had been failing to do. He played, in his words, a very impatient user — and in one afternoon filed eight real bug tickets. None of them were crashes. They were the kind of thing no assertion had been written for: two features prompting for the same permission with two different explanations; an event you could leave but not end, because the end control didn’t exist on the screen where a host would look for it; a status list that only updated if you happened to pull down on it; a prompt that vanished the moment a different user granted their permission.

The uncomfortable part is that the codebase was green. Nineteen hundred automated tests passed the day before. Every one of those eight bugs lived in the gap between “the code does what the spec says” and “a person can tell what is going on” — and our assertions are all on the first side of that gap. Automation verifies the promises you knew to write down. The impatient user discovers the promises you didn’t know you were making.

The economics matter too. The three automated hours produced diagnostic knowledge but nothing closeable. The manual afternoon produced eight tickets, and by midnight eleven fix branches had merged — the human found them faster than the machines could even reproduce them. That ratio won’t hold forever; once a flow is stable, automation is how it stays stable. But at the frontier, where the product is still deciding what it is, the cheapest test harness is a person with no patience and full permission to be annoyed.

The stop-loss went onto our pre-ship question list in one line: has a human walked this flow’s happy path once — before you automated it?