Bounded LLM Repair Loop: Three Independent Limits

U.S. teaching example: Amounts are fictional USD prices, stored as integer cents. This is a U.S. version of the shop exercise, not a currency conversion or a record of a real charge.

A correct proposal can still arrive too late

Suppose a repair loop for a replenishment draft gets two chances to produce a valid order. The first attempt is malformed. The second attempt, generated after feedback, arrives correct in shape but late: the system's deadline expired one second before the response landed. What should the loop report?

Three plausible answers exist, and only one of them is defensible. The loop could report the correct-looking second attempt as accepted, since its content passes every schema and business check. It could report a timeout with no further detail, discarding the fact that content validation happened at all. Or it could report a specific, stable refusal that names the limit that triggered and preserves whatever validation state existed at that moment. This article works through why the third option is the only one that survives contact with a real receiving system, and how to build the three limits — attempts, elapsed time, and external work — as genuinely independent controls rather than one counter wearing three names.

The worked case below extends the bounded replenishment exercise from the Zero-Employee Organization course, which validates a proposal against a trusted inventory snapshot rather than trusting a model to supply both an answer and the policy that approves it. That lesson caps repair at two proposals total. This article adds a deadline and an external-work ceiling alongside the attempt cap, and works out what a receiving team does with the result.

Why one counter cannot do three jobs

An attempt limit answers "how many times did we ask the model to try again." An elapsed-time limit answers "how long has the caller been waiting." A work limit answers "how much has this loop already spent against ledgers outside the process — API calls to a supplier, database reservations, anything with a cost or a side effect beyond CPU cycles." These are different quantities measured in different units: a count, a duration, and a resource total. Collapsing them into one number produces two failure modes.

If you use attempts as a proxy for time, a slow network can let two attempts consume ninety seconds when your service-level target was ten. AWS's Well-Architected guidance on retries makes a related point directly: bound retries, use backoff and jitter, and avoid multiplying attempts across layers, because retry budgets compound when several components each retry independently (AWS Well-Architected: control and limit retries, retrieved 2026-09-10). The guidance also separates retry policy from idempotency and timeout policy explicitly — three concerns, not one dial. If you use elapsed time as a proxy for external work, a loop that spends its whole budget on one long-running database check might still owe a supplier a duplicate reservation from an earlier, technically-within-time attempt. The work ledger needs its own accounting because deleting a timer does not undo a reservation.

The worked case: two attempts, a ten-second deadline, and a work ceiling

Consider the replenishment fixture again: a trusted snapshot with a 3,000-cents budget, needing six vanilla tubs and four strawberry tubs at a combined 2,600 cents. The course lesson validates this deterministically with Pydantic models and a business-rule function that checks snapshot identity, SKU membership, exact replenishment quantities, and budget headroom before returning accepted_draft.

Now bound the repair loop with three independent, hypothetical teaching parameters:

Two hypothetical timing scenarios exercise the boundary:

Scenario A. The first invalid response (wrong type on a quantity field) arrives at second 4. Feedback is generated and a second attempt is requested. That second response is well-formed and passes every business rule, but it arrives at second 11 — one second past the ten-second deadline. The loop has consumed one attempt and has time remaining before it reserves the second request, but not when the response came back.

Scenario B. Both attempts are consumed by second 2, well inside the ten-second window, and both are invalid. The loop must stop here without waiting for the clock, because the attempt limit — not the time limit — is the one that exhausted first.

The distinguishing question is what "stop" means for Scenario A. The fixture author knows the late response contains the right value. The loop need not validate it once time has expired. Does that knowledge count?

The answer is no, and the reason matters more than the rule. A deadline exists because something downstream — a human reviewer, a queued order window, a rate-limited supplier API — stopped waiting. The value of the deadline is that callers can rely on it. If a loop occasionally honors a late-but-valid response anyway, the deadline stops being a deadline and becomes a suggestion that sometimes gets overridden by luck. The terminal result for Scenario A is a stopped state that reports the last known validation outcome (in this case, the first attempt's specific failure code, since the second attempt's result must not be consulted for acceptance purposes) alongside the fact that time, not attempts or work, is what exhausted.

Retaining the validation failure through exhaustion

This connects to the misconception at the center of the bounded-repair design: a repair attempt is still invalid when the limit is exhausted, and the loop must never label it accepted just to produce a tidy finish. In Scenario B, both attempts failed validation. There is no ambiguity about content, only about whether the loop should manufacture a success to avoid returning an error. It should not. The terminal state is repair_exhausted, carrying the specific code from the last validation failure (say, int_type from a malformed quantity field), not a generic timeout or a fabricated accepted_draft.

The course lesson's draft_with_repair function embodies this already for the attempt dimension: it loops for a fixed range, catches ValidationError and PolicyError, and returns needs_review if the range completes without a clean pass. Extending it with a monotonic clock check and a work counter means adding two more early-exit conditions to the same loop, each checked independently before the next attempt is even requested:

class SchemaError(Exception):
    pass

class PolicyError(Exception):
    pass

class WorkLimit(Exception):
    pass

class SupplierBoundary:
    def __init__(self, send, limit=0):
        self.send = send
        self.limit = limit
        self.reserved = 0

    def call(self, payload):
        if self.reserved >= self.limit:
            raise WorkLimit("supplier_work")
        self.reserved += 1  # reserve before dispatch, including failed calls
        return self.send(payload)


def validate_quantity(raw):
    # Small teaching validator, not the full course inventory contract.
    if type(raw) is not int:
        raise SchemaError("int_type")
    if raw != 6:
        raise PolicyError("replenishment_mismatch")
    return {"quantity": raw}


def draft_with_repair_bounded(propose, validate, clock, supplier,
                             max_attempts=2, deadline_seconds=10):
    expires = clock() + deadline_seconds
    attempts = 0
    last_error = None

    def stopped(reason):
        return {"status": "repair_exhausted", "reason": reason,
                "attempts": attempts, "last_error": last_error}

    while attempts < max_attempts:
        if clock() >= expires:
            return stopped("deadline")
        attempts += 1
        try:
            raw = propose(last_error, supplier)
        except WorkLimit:
            return stopped("supplier_work")
        if clock() >= expires:
            return stopped("deadline")
        try:
            draft = validate(raw)
        except SchemaError as error:
            last_error = str(error)
            continue
        except PolicyError as error:
            last_error = str(error)
            return stopped("policy")
        if clock() >= expires:  # validation also consumes time
            return stopped("deadline")
        return {"status": "accepted_draft", "draft": draft, "attempts": attempts}
    return stopped("attempts")

This synchronous example enforces an acceptance deadline. It cannot return on time if propose blocks forever. A production wall-clock response deadline also needs transport timeouts and a supervisor or cancellable task that can stop waiting. Cancellation may leave remote work running, so late responses must be barred from updating a terminal operation record. The local function has no background writer and makes no such cancellation claim.

The injected validator checks one quantity so the limit logic stays visible. In the full course, replace it with the inventory validator. Keep schema failures repairable and policy failures terminal; catching every exception as a repairable format error would hide programming bugs and retry policy refusals.

class FakeClock:
    def __init__(self):
        self.now = 0
    def __call__(self):
        return self.now


def fixture(events):
    clock = FakeClock()
    remaining = iter(events)
    sends = []
    supplier = SupplierBoundary(lambda payload: sends.append(payload), limit=0)
    def propose(feedback, boundary):
        duration, value = next(remaining)
        clock.now += duration
        if value == "try_write":
            boundary.call({"quantity": 6})
        return value
    result = draft_with_repair_bounded(propose, validate_quantity, clock, supplier)
    return result, sends

late, sends = fixture([(4, "6"), (7, 6)])
assert late["reason"] == "deadline" and late["last_error"] == "int_type"
exhausted, _ = fixture([(1, "6"), (1, "6")])
assert exhausted["reason"] == "attempts" and exhausted["attempts"] == 2
blocked, sends = fixture([(1, "try_write")])
assert blocked["reason"] == "supplier_work" and sends == []
accepted, _ = fixture([(2, 6)])
assert accepted["status"] == "accepted_draft"
policy, _ = fixture([(1, 5)])
assert policy["reason"] == "policy" and policy["attempts"] == 1
print("Deadline, attempt, work and policy stops; valid control accepted")

These are simulated seconds advanced by the fixture, not latency measurements. The supplier guard checks its reservation before the mock send function. In a real system, every dispatch path must pass through the guard, with atomic shared reservations if multiple workers can spend the same budget.

What a receiving team needs from this contract

A loop counter and a monotonic timestamp are internal implementation details. They should never reach a person reviewing a stopped order. The interface a receiving team should design against looks like a small, fixed vocabulary:

The client-side failure mode to test for explicitly is a receiving system that treats repair_exhausted as equivalent to accepted_draft because both responses happen to carry a draft field, or because an integration silently defaults unknown statuses to success. A useful regression check is to construct a fake repair_exhausted response with no corresponding order and confirm the client refuses to render a confirmation screen. That check can be written and run today, entirely offline, without a repair loop, a deadline, or a supplier connection existing yet — it is testing the contract, not the implementation.

Separately, whoever owns the loop's internals should verify two things independently rather than trusting one flag: that the attempt limit and the deadline are each individually sufficient to stop the loop (Scenario B above must stop while the clock remains below its limit), and that a late response cannot overwrite a terminal result once returned. A fake-clock fixture, where time is advanced explicitly between assertions rather than read from the wall clock, makes both of these checks deterministic and repeatable rather than dependent on real network timing.

Check the decision

What would falsify this design

The claim here is narrow: three independently-tracked limits, checked before the accept decision rather than folded into one counter, prevent a late-valid response from overwriting a stopped state, and prevent an attempt-exhausted invalid draft from being reported as accepted. This would be falsified by a version of the loop that checks elapsed time only once, before the first attempt, and never rechecks it before returning — that version would let Scenario A's late response through, silently. It would also be falsified by any design that maps repair_exhausted to the same downstream code path as accepted_draft, regardless of how carefully the loop itself is bounded, because the contract failure would happen one layer up. Neither the fake-clock fixture nor the in-memory counters in the code above prove anything about a real supplier's timeout behavior or a real network's latency distribution; they prove the loop's own logic behaves as designed when its inputs are controlled.

The practical next step is to write the fake-clock fixture for both scenarios described above, run it, and confirm the assertions before touching a live model or a live supplier endpoint. Only after that internal contract is verified does it make sense to wire up real transport, at which point the external-work limit — currently zero writes in this teaching version — becomes the one that needs the most careful, separately-tested design.