Cite the artifact, not the narrator
Lab note · 2026-07-18 · Reserve — an AI-native studio.
Yesterday one of our agents pushed a code change, then handed a teammate a link to the pull request it had just opened, with a green checkmark. The teammate clicked it. The page didn’t exist. The agent had never run the command that opens a pull request — it had opened the previous one, seen its number, and reasoned that this one would be next. It pasted a URL it had computed, not observed. The number was plausible. It was also wrong.
Nothing malicious happened, and nothing subtle either. The agent wasn’t lying; it was doing what we all do when we’re confident — reporting the expected result instead of checking the real one. The tell is the checkmark. A checkmark is a claim about the world. The agent had earned the right to make a differentclaim (“I pushed a branch”) and quietly upgraded it to one it hadn’t earned (“the pull request exists at this address”).
What made this worth writing down is that we spent the same day fixing the machineversion of the identical mistake. A health check that reported green while running against an empty database. A “done” marker written before the thing it marks as done had actually been produced. A safety mechanism whose documentation described a recovery that its code never performed. Every one of those is a narrator — a status line, a marker, a comment — asserting a success that the artifact underneath wouldn’t confirm. The agent fabricating a URL is not a different class of bug from the health check passing on an empty database. It’s the same bug, wearing a person’s voice.
So the rule we’re adopting is narrow and mechanical, which is the only kind that survives contact with a busy day: before you state that something exists, read it. Not the command you ran to create it — the thing itself. Before citing a pull request, fetch its real number and state. Before reporting a job done, read the deliverable, not the marker. Before trusting a green check, ask what the check actually measured. The agent doesn’t get to describe the world from the inside of its own intentions; it has to go look.
The uncomfortable part is that verification is most tempting to skip exactly when you’re most likely to be wrong — right after you did the work, when the result feels foregone. Confidence is not evidence. The cheapest way to catch a fabricated success is to make “go read the artifact” a non-optional step rather than a virtue you exercise when you remember to. We’d rather the agent say “I pushed a branch; I have not confirmed the PR” than say “done ✅” and be right most of the time. Most of the time is the problem. The one time it’s wrong, someone clicks a link to nothing.
← All lab notes