Cheaper isn't isolated

Our agent fleet shares one subscription pool. Last week that pool ran dry mid-run, and the way it ran dry taught us to separate three things we’d been treating as one.

The day’s bulk job went first — about a hundred small mechanical rewrites, none individually important. It drank most of the budget. The reasoning-heavy jobs we actually cared about hit the wall later and produced nothing. Not a degraded version. Nothing. The scheduler had no opinion about which mattered, so the budget went to whoever asked first.

Lever one: consumption.Bulk rewriting doesn’t need the expensive reasoning model, so we tiered it — mechanical work to a cheaper model, judgment work unchanged. Burn dropped sharply, quality held where it counts. A good day’s work, and not a reliability fix. The cheaper model draws from the same pool. We didn’t partition anything; we made the bulk job smaller relative to the bucket. That moves the day you hit the wall. It changes nothing about what happens when you do — the low-value job that runs first still eats the budget the high-value job needed. We bought headroom and briefly mistook it for safety.

Lever two: isolation. A separate pool means the bulk job cannotstarve the critical one at any level of spend. We’d left a seam for exactly this and then found we couldn’t use it: one subscription, nothing to hang on the seam. The partition was the right answer and we couldn’t afford it.

Lever three: admission control— the boring one, and the one we should have reached for first. If you can’t partition the pool, you have to order it. Calls become sacrificial by default; a short list of genuinely important ones get marked protected; when the breaker trips, sacrificial work stops instantly while protected work gets a single probe after a cooldown to test whether the limit has lifted. That’s written and in review as of this note — not yet merged, so on today’s evidence the ordering is a claim about a diff, not about a running system.

Three things we learned building it, all of which generalize:

The safety mechanism was lying.Our circuit breaker’s docstring promised that any success resets it — that it auto-recovers when quota returns. That sentence had never once been true: the tripped breaker returned early without ever attempting a call, so no success could ever be recorded to reset it. One transient rate-limit killed the rest of an hour-long run silently. The comment described the behavior we wanted; nobody had checked whether the code agreed. A breaker that can’t reset isn’t a breaker, it’s a fuse — and we’d documented it as a breaker.

Importance and cost are different axes.The tempting shortcut is to sacrifice the expensive calls first. But our most expensive job is sacrificial (it can run tomorrow) and one of our cheap ones isn’t. Quality tiering and sacrifice ordering are orthogonal; collapsing them into one number gets you a system that protects the wrong work confidently.

Be reactive, not predictive.We don’t estimate the day’s burn in advance — the ordering only engages once a real scarcity signal appears. On abundant days the mechanism may as well not exist, so it can never cut something that would have finished fine. Predicting scarcity means being wrong in both directions; reacting to it means being wrong in neither.

And the honest remaining gap: the breaker’s state lives in one process’s memory, and three processes draw on the same pool without seeing each other. Each one now orders its own queue beautifully and is blind to the other two. At the pool level, we are still first-come, first-served. You can only order a queue you can see — which is the partition problem again, one level down, waiting for us.