Reserve an Agent's Spending Before Each Request, Not After

Dollar amounts on this page are in USD.

Two workers, one last slot

You give an eval run a budget and start 16 workers. Near the end, the budget has room for exactly one more call. Two workers reach it at the same moment. Each reads the spend so far, sees room, and sends its request. Now two calls are out against room for one. One of them times out after the provider has started generating, so it may have been billed even though no response came back.

Most first attempts at agent cost control are a counter: add each call's cost, stop at the limit. The counter lives in one process's memory. It works for one worker running calls one at a time. It fails as soon as there is a second worker, a call that fails without the counter noticing, or a restart that forgets what was spent. Bounding money across concurrent workers and interrupted calls needs a ledger that every worker writes through, and a reservation made before each call.

Chapter 11 of the book reserves spending in the same transaction that records an approval, and Chapter 19 derives what a model call and an agent loop cost. This article adds the per-call maximum-charge arithmetic, the overshoot you get without reservation, settlement from real usage, and a concurrency test with its receipt.

Token caps are not a money limit

Three limits often get treated as one. A request-size cap bounds what you send. An output-token cap, max_tokens, bounds what the model generates. A dollar ceiling bounds what you pay. Tokens become money only through a price per token, which differs by model, by input versus output, and by cache use. Anthropic's prompt caching documentation, for example, bills cached input differently from fresh input (Anthropic, "Prompt caching"). A system that counts tokens moved, without knowing which were cache writes or reads, can get the money wrong with perfect token accounting.

Providers also enforce limits of their own. Anthropic's API has a monthly spend cap per usage tier, and lets you set a lower spend limit for the organization or a workspace. Reaching a limit you set returns a 400 error; reaching the tier cap returns a 429 that retrying will not clear (rate limits). Those limits protect the organization's month. They do not stop one sweep from consuming a month's budget in an afternoon. That is the job of a per-run ledger.

The maximum charge of one call

Before a call is sent, you can bound what it can cost:

max_charge = input_bound × input_rate + max_tokens × output_rate + other billed categories

Use integer microdollars (millionths of a dollar) so the arithmetic is exact. With a hypothetical tariff of 2 microdollars per input token and 8 per output token (2 and 8 dollars per million), a call bounded at 2,000 input tokens and 500 output tokens has

max_charge = 2,000 × 2 + 500 × 8 = 8,000 microdollars = $0.008

If 12,000 microdollars remain, one such call fits and two do not.

Three details keep this bound honest. The input bound must cover the request as actually serialized, including system prompt, tools and history. Anthropic's token counting endpoint returns the input tokens for a request before you send it, but its documentation calls the count an estimate that "might differ by a small amount" from actual use (token counting), so reserve with a margin. If caching is on, price the input at the highest rate that could apply, such as the cache-write rate, not the rate you hope a cache hit gives. And include any separately billed tools; if you cannot bound a category, call the control an estimate, not a ceiling.

View full-size diagram

What happens without a reservation

Suppose k workers each check "spent + max_charge ≤ limit" against the same stale reading, and each passes. All k send. If the remaining budget covered only one call, the overshoot can reach

overshoot ≤ (k − 1) × max_charge

With 16 workers and the $0.008 call, that is $0.12 per race, which is small. The same race with a long-context agent call is not. Using the same hypothetical rates, a call bounded at 150,000 input and 8,000 output tokens has a maximum charge of 150,000 × 2 + 8,000 × 8 = 364,000 microdollars, $0.364, and 15 extra calls cost about $5.46 in one race. Races happen at every boundary: the end of a run, a budget shared across runs, a quota shared across a team.

Reserve before you call

The fix is a reservation that commits durably before the network call. A worker opens a transaction that takes the ledger's write lock, reads what is already committed, checks that the new maximum charge fits, inserts a reservation row, and commits. Only then does it build the request. A second worker blocks on the lock, then reads the updated total and is refused before sending anything.

This site's own editorial review launcher uses this pattern for a batch of review calls (a case study, inspected on September 10, 2026). It reserves calls, request bytes and output tokens in SQLite under BEGIN IMMEDIATE before any request is built, and never refunds a reservation after a failed or interrupted call. It bounds calls and tokens, not dollars. The code below adds the money bound.

View full-size diagram

The ledger, and a race to test it

The code uses Python's standard library, SQLite and threads, and runs on Colab's Python 3.12. A reservation holds its maximum charge until it is settled from authoritative usage.

import os
import sqlite3
import tempfile
import threading
import time

INPUT_RATE, OUTPUT_RATE = 2, 8  # microdollars per token, a hypothetical tariff


def max_charge(input_tokens_bound, max_tokens):
    return input_tokens_bound * INPUT_RATE + max_tokens * OUTPUT_RATE


def make_ledger(path, limit):
    db = sqlite3.connect(path)
    db.executescript("""
        CREATE TABLE budget (id INTEGER PRIMARY KEY CHECK (id = 1), limit_micro INTEGER NOT NULL);
        CREATE TABLE reservations (
            operation_id TEXT PRIMARY KEY,
            reserved_micro INTEGER NOT NULL,
            settled_micro INTEGER            -- NULL until authoritative usage arrives
        );
    """)
    db.execute("INSERT INTO budget VALUES (1, ?)", (limit,))
    db.commit()
    db.close()


def committed(db):
    # A settled reservation counts its actual charge; an unsettled one counts its maximum.
    return db.execute(
        "SELECT COALESCE(SUM(COALESCE(settled_micro, reserved_micro)), 0) FROM reservations"
    ).fetchone()[0]


def reserve(path, operation_id, charge):
    db = sqlite3.connect(path, timeout=30, isolation_level=None)
    try:
        db.execute("BEGIN IMMEDIATE")  # take the write lock before reading
        limit = db.execute("SELECT limit_micro FROM budget").fetchone()[0]
        if committed(db) + charge > limit:
            db.execute("ROLLBACK")
            return False
        db.execute("INSERT INTO reservations VALUES (?, ?, NULL)", (operation_id, charge))
        db.execute("COMMIT")
        return True
    finally:
        db.close()


def reserve_unguarded(path, operation_id, charge):
    db = sqlite3.connect(path, timeout=30)
    try:
        limit = db.execute("SELECT limit_micro FROM budget").fetchone()[0]
        room = committed(db) + charge <= limit
        time.sleep(0.001)  # the gap between check and use
        if not room:
            return False
        db.execute("INSERT INTO reservations VALUES (?, ?, NULL)", (operation_id, charge))
        db.commit()
        return True
    finally:
        db.close()


def settle(path, operation_id, input_tokens, output_tokens):
    db = sqlite3.connect(path, timeout=30)
    with db:
        db.execute(
            "UPDATE reservations SET settled_micro = ? WHERE operation_id = ? AND settled_micro IS NULL",
            (input_tokens * INPUT_RATE + output_tokens * OUTPUT_RATE, operation_id),
        )
    db.close()

The race starts eight threads behind a barrier, each with its own connection, against a ledger with 12,000 microdollars left and a maximum charge of 8,000. Exactly one should be admitted.

def race(fn, workers=8, limit=12_000, charge=8_000):
    path = os.path.join(tempfile.mkdtemp(), "ledger.db")
    make_ledger(path, limit)
    barrier = threading.Barrier(workers)
    admitted = []

    def worker(i):
        barrier.wait()
        if fn(path, f"item-{i}", charge):
            admitted.append(i)

    threads = [threading.Thread(target=worker, args=(i,)) for i in range(workers)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    return len(admitted)


print("max charge:", max_charge(2_000, 500))
trials = 50
guarded = [race(reserve) for _ in range(trials)]
unguarded = [race(reserve_unguarded) for _ in range(trials)]
print("guarded:   admitted exactly one in", guarded.count(1), "of", trials, "trials; max", max(guarded))
print("unguarded: admitted more than one in", sum(n > 1 for n in unguarded), "of", trials, "trials; max", max(unguarded))

# Settlement frees the difference between the maximum and the authoritative usage
path = os.path.join(tempfile.mkdtemp(), "ledger.db")
make_ledger(path, 14_000)
print(reserve(path, "item-1", 8_000), reserve(path, "item-2", 8_000))
settle(path, "item-1", 1_500, 300)  # 1,500 * 2 + 300 * 8 = 5,400
print(reserve(path, "item-2", 8_000))

One run on an Apple silicon Mac (OS 26.6.2), Python 3.12.13 and SQLite 3.50.4 printed:

max charge: 8000
guarded:   admitted exactly one in 50 of 50 trials; max 1
unguarded: admitted more than one in 50 of 50 trials; max 8
True False
True

With the reservation, exactly one of eight workers was admitted in all 50 trials. Without it, more than one was admitted in every trial, and in at least one trial all eight were. That is the (k − 1) overshoot at its worst. Your counts for the unguarded version will vary with your machine's scheduling; the guarded count should not. The 1-millisecond sleep widens the gap between check and use so the race shows up reliably. Without it, the race is rarer, not absent.

The last two lines show settlement. With a 14,000 limit, the first reservation fits and the second does not (16,000 > 14,000). After the first call's authoritative usage settles it to 5,400, the second fits (13,400 ≤ 14,000).

Settle from evidence, never from hope

Holding every reservation at its maximum is safe and wasteful. If calls use on average 40% of their maximum, then while k calls are in flight about 0.6 × k × max_charge of budget is held but will never be spent. That is the cost of safety, and it is bounded by concurrency: with 16 workers and a maximum of 8,000 microdollars, at most 128,000 microdollars ($0.128) is held at once, whatever the size of the run. Settlement returns the difference once you know it.

Settle only from authoritative usage: the usage block in the provider's response, or the provider's usage records for that request. An interrupted call has no such evidence. Its request may never have left, or the provider may be generating a response nobody will read. Keep its reservation at the maximum, record it as uncertain, and reconcile it later against the provider's usage records if you can match them. A timeout is not evidence that nothing was billed.

Check the decision

Retries sit on top of the ledger

AWS's Well-Architected guidance recommends bounding retries, applying backoff and jitter, and keeping retries from multiplying across layers (AWS, "Control and limit retries"). That answers a different question. A retry cap stops one call from looping forever. A reservation ledger stops concurrent workers from jointly spending the same budget. Each worker can stay within its own retry cap and the fleet can still overspend. The circuit breaker article derives how retries multiply; every one of those retries needs its own reservation.

Why this matters in a lab

Inference is one of the largest line items in applied research: evals over tens of thousands of items, sweeps across models, agents that run for hours. A research engineer is expected to say before a run what it can cost at most, and to make that true under concurrency, crashes and retries.

An interviewer might ask: "You have $500 for an eval across three models and 16 workers. How do you guarantee you don't exceed it?" A strong answer computes the maximum charge per call from bounded input and max_tokens, with margin on the token count and the worst-case cache rate. It reserves atomically before each send, derives the (k − 1) overshoot of the naive approach, settles only from provider usage, routes SDK retries through the ledger, and shows a concurrency test like the one above. For the cost side, Chapter 19 prices a loop whose input grows with the square of its length.