Week 0: The Question You Should Be Asking Before You Fund or Build One
Prerequisite: none. This is a short framing lesson before Week 1, not a technical lesson, and not required reading if you'd rather jump straight in. After this lesson, you can: name the one question this course spends nine lessons answering, and know within a paragraph whether you're the funder this course was built for, the engineer, or both.
Here's a bet worth making before you commit 82 minutes to this course: if you've watched an AI agent demo in the last year and thought "that's ready," you were probably wrong, and it probably wasn't the model's fault.
The gap this course exists to close
Somewhere between 17% and 42% of companies abandon most of their AI initiatives before they ever reach production, and that number got worse, not better, over the last year. S&P Global's 451 Research surveyed over 1,000 professionals across North America and Europe for its Voice of the Enterprise: AI & ML 2025 report and measured that climb directly (Lesson 01 walks through the full figure). Almost every postmortem blames the model. Almost none of them are right. A demo and a production system run the same model. What's different is everything wrapped around it, the harness, and nobody taught you to look there, because the harness never shows up in a thirty-second demo.
That's the whole course in one sentence. The model is roughly constant. The harness is where projects live or die. Right now you have no framework for evaluating it. If you fund AI projects, you're grading pitches on the wrong axis. If you build them, you don't yet have the shared vocabulary for the thing that's actually failing.
The mistake this course corrects
If you currently believe an agent demo predicts production behavior, you're not being naive. It's the intuitive read, and almost everyone starts there. The correction isn't "demos lie." It's narrower and more useful than that: a demo tests 2-3 steps on a clean, curated happy path, and production runs ten or more steps against messy real input, rate limits, and edge cases nobody scripted. Chain a 95%-reliable step ten times (0.95¹⁰) and the task succeeds about 60% of the time. Chain it twenty times (0.95²⁰) and that drops to roughly 36%. No single step got worse. The math just caught up with the chain. That's not a model problem you can prompt your way out of, and Lesson 01 walks through exactly why.
What you'll be able to do that you can't do now
Right now, if someone shows you an agent demo, you have one instinct: is the model good? After this course, you'll have a second, better-calibrated one: is the harness real? Concretely, you'll be able to:
- ask five plain-language questions that expose a weak AI project before you approve its budget, no technical background required
- name the six-category failure taxonomy behind harness engineering, so "it broke in production" stops being a mystery and starts being a diagnosis
- place any agent you're evaluating against one of four real architectures (research agents, workflow agents, decision-support agents, or policy-enforced conversational agents), and know what that architecture specifically demands before it ships
- walk out with a five-hour plan to sketch a working harness skeleton for your own project, not just an opinion about someone else's
| Before this course | After this course |
|---|---|
| One instinct: is the model good? | Two instincts: is the model good, and is the harness real? |
| A smooth demo reads as "ready" | A smooth demo reads as "untested at production load" |
| "It broke in production" is a mystery | "It broke in production" maps to one of six named failure categories |
| An opinion about someone else's architecture | A five-hour plan to sketch your own |
Who this is for
The course splits cleanly into two doors, and you can walk in either one.
No programming background required. Built for anyone who funds, greenlights, or advises on AI projects and is tired of grading demos on vibes.
Assumes comfort reading YAML and simple Python. Four real architectures worked through with manifests, verifiers, and a named case study on Rasa's own CALM system.
Neither door requires you to have built an agent before.
The Taco Bell drive-through that crashed on 18,000 cups of water, the arithmetic behind why a 95%-reliable step fails one time in three after ten hops, and the reframe that makes the rest of this course make sense.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
