Don't ask the agent to grade itself — run a competition

We spent a day designing a “review officer” for an agent that makes daily calls — the kind of agent whose output you’re tempted to trust because it writes a confident paragraph. The first three designs all made the same mistake, and the mistake is instructive.

Version 1 was a self-critique loop:ask the agent to grade its own past calls and adjust. The problem is obvious once you say it out loud — the same model that made the call is now the judge of the call, and a model’s stated confidence is plausibility, not probability. Asking it to introspect just launders the original bias through a second prompt.

Version 2 added structure — an attribution tree, a calibration curve, a lessons library. Better telemetry, same flaw: it was still the agent auditing the agent. More dashboards on a conflicted witness.

Version 3 threw out introspection entirely. The reframe that worked: the system doesn’t reflect on itself — the system runs a race. Three frozen ledgers:

• a shadow ledgerwith zero discretion (the mechanical baseline — what the pipeline would do with no “smart” judgment on top),
• the main ledger(the agent’s actual discretionary calls), and
• a benchmark ledger (buy-the-index-and-hold, do nothing).

A single deterministic settlement engine scores all three, append-only, every day. Three rules became constitutional: everything that influences a decision leaves a trace; every number is settled before it’s consumed; every lesson must survive the competition, not a self-review.

The agent’s worth is no longer something it asserts — it’s a difference between ledgers. Main minus shadow = what the judgment actually added. Shadow minus benchmark = whether the mechanical layer beats doing nothing. If the discretion ledger doesn’t beat the no-discretion one, the “smart” layer is decoration, and the numbers say so without anyone having to be persuaded.

Why this generalizes. When we scanned recent literature to stress-test the design, the three-frozen-ledgers shape turned out to be isomorphic to the Darwin Gödel Machine— a frozen archive plus empirical selection rather than an agent editing its own beliefs (the DGM work reported SWE-bench climbing 20% → 50% under that regime). And the “LLM proposes, a deterministic engine settles, an append-only log records” split has out-of-sample backing in the hypotheses-to-factors line of work. Different domain, same lesson: you don’t make an agent trustworthy by making it more self-aware. You make it trustworthy by putting it in a contest it can’t referee.

The uncomfortable corollary we wrote down too: a competition can be gamed by proposing many variants and cherry-picking winners (p-hacking by another name), so the acceptance bar for a challenger has to rise with the number of variants tried. Accountability isn’t a feature you add at the end — it’s a structure you’re either inside or outside of.