Week 0: Why Agent Systems Fail, and How a Lab Would Find Out
Prerequisite: none. This is a short framing lesson. After this lesson, you can: state the question this course answers, and say which lessons matter most for the role you are preparing for.
An agent run is a chain of steps: read the task, call a tool, read the result, decide, call the next tool. When the chain breaks, the easy explanation is "the model wasn't good enough." Sometimes that is true. Often the failure lives in the system around the model: a tool that returned stale data, state that did not survive a crash, a retry that charged twice, a context window full of noise, or an evaluation that measured the wrong thing. That system is the harness, and this course is about finding out which part of it failed.
The question the course answers is the one an applied-AI or research team asks after every surprising result: why did this agent fail, and how would we have caught it before a user did?
The arithmetic that frames the course
Take an agent whose every step succeeds with probability 0.95. If the steps fail independently and the task needs all of them, a 10-step task succeeds with probability 0.95¹⁰ ≈ 0.60, and a 20-step task with 0.95²⁰ ≈ 0.36. No step got worse. The chain got longer. Lesson 01 derives this properly, shows where the independence assumption breaks, and shows what a verifier plus a retry does to the number.
That one line of arithmetic explains why a demo, which runs a few steps on a prepared input, tells you little about a 20-step task on messy input. It also tells you where to look: at the steps, at how failures are detected, and at how success is measured.
Who this course is for
You are preparing for a member-of-technical-staff role in applied AI or research, or you already work on a large AI team and want the failure-analysis habits in one place. You may be:
- an engineer building with models, heading for an applied-AI role;
- a researcher or PhD student who runs experiments and evaluations;
- someone inside a lab or large AI team filling a gap;
- a student or early-career engineer.
The course asks for comfort reading Python and YAML and for basic probability. It derives everything else on the page.
How the ten lessons fit together
Reliability that compounds across steps, with error bars. How to read a public postmortem for its root cause and for the evaluation that missed it. How failure analysis shows up in an interview loop and a take-home work sample.
A failure taxonomy organized by harness layer (tools, state, retries, context, evaluation, oversight), then three agent shapes worked through with manifests, deterministic verifiers, review gates and calibrated graders.
The failure catalog in one table, a five-hour work sample you can show, and a fourth shape: a conversational workflow where the model parses and phrases and code makes every policy decision.
What you will have at the end
A work sample: a manifest schema for one unit of agent work, a deterministic verifier, one human-review gate placed on purpose, one failure mode with a defense you tested, and an evaluation reported with honest error bars. The companion lab, an offline evidence gate in the public profrodai-resources repo, runs on Python's standard library with no model, key or network.
The course pairs with the book. Chapter 3 builds the agent loop and its reliability math from scratch, Chapter 16 covers error bars and paired comparisons, and Chapter 20 covers acceptance and honest reports.
Derive p to the n, put an error bar on it, and see what a verifier plus a retry does to a 10-step task.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
