The Demo Worked, the Project Failed
Prerequisite: none — this is the course's opening lesson. After this lesson, you can: separate "is the model capable?" from "is the harness built?" when you evaluate any AI agent pitch, and explain in one sentence why a working demo predicts almost nothing about production survival.
Almost everyone evaluating an AI agent project (funder, advisor, the person newly asked to "own AI") is grading the wrong thing. They watch the demo, judge the model, and sign or decline based on how smart it seemed. That's the wrong question. The right question is what's built around the model, because that's what actually decides whether the project survives contact with production.
The gap has a name, and it isn't intelligence
Taco Bell's drive-through AI has been live at more than 500 locations since 2023 and has processed over two million orders. It is not a toy project run by an unserious team. It is also the system one customer crashed by ordering 18,000 cups of water in a single session, in a video that has since been watched millions of times. Taco Bell's Chief Digital and Technology Officer, Dane Mathews, told the Wall Street Journal in August 2025: "sometimes it lets me down."
Hold that contradiction, because it's the whole lesson in miniature: a system that mostly works, built by a competent team, failing in public in a way nobody demoed. That is not a Taco Bell story. It is the AI agent story.
It's also, increasingly, the rule rather than the exception. S&P Global's 451 Research surveyed 1,006 professionals across North America and Europe for its Voice of the Enterprise: AI & ML 2025 report and found that the share of companies abandoning most of their AI initiatives before production rose from 17% to 42% in a single year, with roughly 46% of proofs-of-concept scrapped along the way. This lesson leads with that number deliberately. You'll also hear that "95% of pilots fail," a figure that traces back to MIT research, but that number measures "no provable profit impact," which is a measurement problem as much as a technology one, and it collapses under scrutiny faster than it should for how often it gets repeated. The S&P figure has a named, transparent sample behind it. Lead with 42%. It's damning enough on its own.
The MIT figure gets quoted more because it's scarier, but it measures whether a pilot produced provable profit impact, a much fuzzier bar than whether the system worked. S&P Global's 42% is the more defensible number: a named survey, a named sample, a one-year comparison. When a number is doing a lot of persuasive work, check what it actually measured before repeating it.
It is not a capability problem
The demo is a controlled environment. Production is not. The model itself is roughly constant across both settings. What changes is everything else: the engineering and operations built around it.
| The demo | Production | |
|---|---|---|
| Inputs | Clean, curated | Messy, real |
| Steps | 2–3, the happy path | 10 or more |
| Conditions | Nothing unexpected | Edge cases, rate limits, compliance |
| Stakes | Nothing real | Real customers, real data, real liability |
This isn't one contrarian take. S&P Global's own survey, cited above, is corroborated by other industry research pointing the same direction. The most expensive mistake a funder can make is to evaluate the brain and ignore the body.
To see why, it helps to be precise about what an agent actually is, because the failure mode is baked into the definition. A chatbot is one hop: you ask, it answers. If the answer is wrong, it's one wrong sentence: you read it and move on. An agent is a chain. It looks something up, decides, calls a tool, takes an action (issues a refund, say), and keeps going. Two things follow from that. First, it's now doing things in the world, not just talking. Second, each step depends on the one before it, so a wrong step early means everything after it is wrong too.
That chaining is what makes agents useful, and it's exactly what makes them fragile, because reliability multiplies down a chain.
Sounds excellent in isolation: a single lookup or tool call that's right 19 times out of 20.
The task now succeeds only about 60% of the time (0.95¹⁰ ≈ 0.60). This is a realistic production workflow.
Down to roughly 36% (0.95²⁰ ≈ 0.36). It's pure multiplication. No single step got worse.
A demo shows you two or three steps on the happy path, so it looks 95%-solid. Production runs ten or more steps over messy inputs, so the same agent collapses toward 60% or 36%. The demo isn't lying. It's just short.
The engine, and the rest of the car
The metaphor worth carrying into every AI conversation you have: the model is the engine. Powerful, cheap, increasingly rentable, and just one part of the whole vehicle. Everything wrapped around it is the harness: grounding (answering from your facts instead of the model's memory: "look it up, don't recall it"), tool and system integration (how it actually issues the refund or reads the order), monitoring and cost control (seeing what it did and what it spent), evaluation (automated tests of output quality, unit tests for behavior), guardrails and confidence thresholds (hard limits, and the discipline to stop when unsure), and human escalation (handing a hard case to a person, with context). None of those six show up in a thirty-second demo. All of them decide whether you ship.
A brilliant engine bolted to nothing is a demo, not a product. The harness (brakes, steering, dashboard, fuel) is what turns it into something you'd put a customer inside.
There's real evidence behind the claim that the harness carries the leverage, not the model. Researchers at Stanford and Tsinghua took the identical model (same weights) and varied only the engineering wrapped around it. Performance swung by up to 6x. On a separate standard engineering benchmark reported by Hugging Face, scaffold-only changes, with no change to the model at all, added 10 to 20 points. The question that predicts whether a project ships was never "which model are you using?" It's "what have you built around it?"
In money, legal, and health, the model still sets hallucination rate, reasoning depth, and cost, and those three are everything in those domains. Measured hallucination rates run from roughly 4% to over 60% depending on task and method. That width is the point: never trust a single tidy number from a vendor. The model is commoditized for raw capability, and the harness is where projects win or lose. But a good model is necessary. It's just no longer sufficient, and never "solved."
What to do with this
Before you evaluate the next AI pitch, separate the two questions a funder is actually being asked to answer.
Is the model capable enough? In 2026 this usually answers itself: frontier-class models from multiple labs are increasingly interchangeable, and cost per word keeps falling.
Is the harness built? Almost nobody asks this one, because it's the part that never makes it into the demo. It's also the question that actually predicts whether the project ships.
The next lesson turns this framing into something you can use in the room: five specific warning signs, each with a plain-language question that exposes in seconds whether a project's harness is real or whether you're being shown an engine with no car around it.
Five warning signs, and the one question for each that a real team answers in seconds and a hype-driven team dodges.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
