U.S. teaching example: Amounts are fictional USD prices, stored as integer cents. This is a U.S. version of the shop exercise, not a currency conversion or a record of a real charge.
The five-tub proposal that keeps almost working
Picture a small ice cream shop's replenishment assistant. Lucy has two tubs of vanilla on hand and wants eight, so the correct order is six. A model asked to draft that order comes back with five. Every field in its JSON is well-typed: the SKU exists, the quantity is a positive integer, the schema validates cleanly. And the order is still wrong. Five is not six.
This is the moment many people building agent-facing systems get stuck on. It's tempting to feed that mismatch back to the model as an error and let it try again, the same way you'd handle a malformed field. But there is a real difference between a proposal that fails to parse and a proposal that parses fine but violates a rule enforced by the application. Conflating those two things is where validation feedback design usually goes wrong, and it's the exact question this article works through: what do you say back to the model when it's wrong, without saying so much that you hand it the keys to decide whether it was wrong at all?
Two kinds of failure, one interface
Schema validation checks shape: is quantity an integer, is sku a string, does the object have all its required keys. Pydantic's strict mode, documented here, tightens this further by refusing quiet type coercion, so a quantity arriving as the string "6" fails instead of silently becoming the integer 6. That distinction matters because a model that sends "6" made a formatting mistake, not a business decision, and a bounded repair may be appropriate when validation has no side effects. It still costs a model call.
Business-rule validation checks something else entirely: does this well-formed proposal match what the trusted system of record says should happen. Five vanilla tubs is a perfectly shaped OrderLine. It is also arithmetically wrong, because required = max(0, target - on_hand) computes to six, not five. That computation lives in your application, against data the model never touched directly. The teaching example behind this article, from the ZEO inventory course, builds exactly this boundary: a trusted snapshot with a 3000 cents budget, a proposal requiring six vanilla tubs and four strawberry tubs at 2600 cents total, and a validator that raises a named PolicyError when the proposal doesn't match the required lines or exceeds budget.
The two failure kinds need different feedback, because they invite different repairs. A schema failure is a request to reformat. A policy failure is, at most, a request to reconsider within the same rules, and sometimes a flat refusal that no regeneration should override.
What goes into the message back to the model
Anthropic's guidance on writing tools for agents makes a point relevant here: how a tool describes its failures shapes how reliably an agent uses it (Anthropic, "Writing tools for agents"). A vague or oversharing error message either gives the model nothing to act on or gives it too much, including facts it should never have been able to see. The design task is to pick, for each failure code, exactly what crosses that boundary.
Consider three failures against the shop fixture:
| Failure | What happened | Safe to send back | Never send back |
|---|---|---|---|
| Malformed quantity | "quantity": "6" instead of 6 | Field path lines[0].quantity, code int_type | Raw rejected input or an unnecessary stock record |
| Missing required shortage | Proposal omits strawberry entirely | Code replenishment_mismatch, the field path for lines | Raw internal records or instructions to change the replenishment policy |
| Over-budget draft | The original requirements cost 2600 cents; adding a genuine shortage of four Lime tubs at 200 cents raises the required total to 3400 cents | Code budget_exceeded | An instruction to alter required quantities merely to obtain approval |
The malformed type is repairable under this course contract. The missing shortage is a business-rule mismatch and goes to review. Telling the model its quantity field failed type validation, and where, lets it resend a corrected integer without ever learning what the "right" quantity is supposed to be from your side; it still has to compute that from its own inputs. The third row is different in kind. If you send back "over budget, try a cheaper combination," you have effectively asked the model to search for any number that satisfies your gate, which turns the validator into a puzzle for the model to solve rather than a policy your application enforces. The correct move is to report budget_exceeded and stop, or escalate to a human, never to coach the model toward the budget's actual value.
A budget need not be secret to be enforceable. The application must check required quantities and the trusted budget regardless of what the model knows. This example sends only a code to minimize disclosure; secrecy is not its authorization mechanism. A different task, such as choosing optional items within a public budget, could legitimately invite a cheaper proposal.
Building the feedback function
Here is a minimal, self-contained version in standard Python, close in spirit to the validator from the course lesson, reduced to show the feedback shape rather than the full inventory model.
from pydantic import ValidationError
class PolicyError(ValueError):
def __init__(self, code):
self.code = code
super().__init__(code)
def feedback_for(error):
"""Map an exception to feedback the model can act on,
without leaking trusted values."""
if isinstance(error, PolicyError):
# Policy failures: name the code only. No thresholds, no arithmetic.
return {"kind": "policy", "code": error.code}
if not isinstance(error, ValidationError):
return {"kind": "internal", "code": "unsupported_error"}
fields = {"lines", "quantity", "sku"}
issues = []
for item in error.errors(include_input=False, include_context=False)[:5]:
# Unknown keys can themselves contain untrusted text.
safe_path = [part if type(part) is int or part in fields else "unknown"
for part in item["loc"]]
issues.append({"field": safe_path, "code": item["type"]})
return {"kind": "schema", "issues": issues}
# Hypothetical calls, illustrating shape only:
# feedback_for(PolicyError("budget_exceeded"))
# -> {"kind": "policy", "code": "budget_exceeded"}
#
# feedback_for(some_validation_error_for_bad_quantity_type)
# -> {"kind": "schema", "issues": [{"field": ["lines", 0, "quantity"], "code": "int_type"}]}Notice what the function withholds by construction. A PolicyError carries only a code, no message string built from live inventory numbers. A schema ValidationError is filtered with include_input=False, include_context=False, so the rejected value itself, which might contain something you don't want echoed back into a future prompt, never surfaces in the feedback. Even structural paths need filtering when an unknown input key could contain arbitrary text. The function limits the issue count and maps undeclared path names to unknown; policy values remain application-owned whether or not they are visible to the model.
The diagram's branch point is the whole design. Sanitized schema feedback can invite one repair in this example. Everything right of the policy check either flows back as a bare code with no numbers attached, or does not flow back to the model at all and instead goes to a human reviewer.
Bounding the loop so refusal stays a refusal
A repair loop needs an attempt limit, and the limit has to distinguish "try again" from "stop." Two attempts total is a reasonable bound for a schema-level fix: the model resends its proposal once with corrected structure. A policy failure like budget_exceeded or a stale snapshot ID should end the loop immediately, not consume a retry, because no amount of regeneration changes an authorized limit or a database's current state.
Classify the exception as schema or policy before building any feedback string; the two paths never share a message shape.
For schema failures, extract only the field path and Pydantic's error type code, filtering out the rejected input value itself.
For policy failures, return the bare code and nothing else, then end the loop or route to human review rather than issuing a second proposal request.
This is what the course lesson's fixture proposer demonstrates: a stubborn proposer that repeats the same malformed quantity twice never gets a third attempt, and a proposer facing an over-budget snapshot gets no repair invitation at all, only a needs_review status after its first proposal. Those are counted outcomes inside one local test file. They show the loop's logic does what it's supposed to on that fixture; they say nothing about how an external model provider would behave against a live endpoint, and nothing about network timeouts, concurrent stock changes, or rate limits, which sit outside this validator entirely.
Versioning the codes so meaning doesn't drift
A second, easy-to-miss failure mode: a code like budget_exceeded starts as a policy refusal, and six months later someone "helpfully" changes its handling so that it also fires for a repairable rounding difference. Now a client built to treat budget_exceeded as terminal silently retries something it shouldn't, or a client built to retry a schema code silently gives up on something fixable. The fix is to treat the mapping from code to retry behavior as a versioned contract, not an implementation detail.
A concrete shape for that contract: {"contract_version": "validation-errors-v1", "code": "quantity_invalid", "field": "lines[0].quantity", "retry_class": "repair_input"}. If quantity_invalid ever needs to mean something that should stop retries instead of inviting them, that change ships as validation-errors-v2, with a migration note, and existing clients keep their old behavior until they explicitly adopt the new version. An unrecognized code should produce an explicit unsupported_error response, never a silent success. This is a proposed interface shape for teaching purposes; no shipped public API is being described here, and building it for a real integration means writing the version negotiation and testing an old client against the new codes before calling it done.
Check the decision
What would show this design is wrong
This whole approach rests on a claim worth stating so it can be tested: that this classifier and the surrounding loop preserve the declared repair policy. A pass/fail budget oracle can reveal a threshold across repeated queries even without numeric feedback. The two-attempt loop bounds one invocation, not all future requests by the same caller. If the threshold is confidential, analyze cross-request access and inference separately; hiding the number in one error message is insufficient. It would also fail if a legitimate schema fix, like a corrected quantity type, kept getting misrouted as a policy refusal because of a bug in the classification step, silently converting repairable errors into unnecessary human escalations. Both are testable with fixture proposers like the one in the course lesson: run enough varied bad proposals through the loop and check whether attempt counts and final statuses land where the code paths say they should. That is a local proof about your validator's logic. It is not a proof about how any particular hosted model would behave when it receives these codes in a live prompt, and that gap should stay explicit in any writeup of results.
Where to take this next
The worked validator here uses a single-shop, single-currency fixture on purpose, so the schema/policy split stays visible without extra noise. Real systems add currency conversion, concurrent stock writes, and multi-step approval, each of which needs its own bounded rule rather than an ad hoc extension of this one. The ZEO inventory course builds the surrounding pieces, including the stale-snapshot check and the exact-budget boundary test, in the same fixture used above.
See the complete validator, the exhaustion tests, and the Lime transfer exercise that extends this same budget boundary.
