Skip to content

The Question Nobody Asks Before Building

Before you start

Prerequisite: none, this is the course's opening lesson. If you've taken Tools, Memory, and Multi-Agent Systems, its lesson 01 is useful background but not required here. After this lesson, you can: name the two questions (who is waiting for this agent, and who decides what happens next) that determine an agent's architecture before a line of code is written, and explain why skipping them, not model quality, is what sits behind most agentic AI projects that get cancelled.

Why 40% of these projects get cancelled

Ask a team why their agent project got cancelled and you'll almost never hear "the model wasn't good enough." You'll hear about scope creep, unpredictable behavior, a demo that impressed everyone and a production system that impressed no one. Kate Jensen, Head of Americas at Anthropic, put a number on the pattern in February 2026: "2025 was meant to be the year agents transformed the enterprise. The hype was premature, not a failure of effort, but a failure of approach." Gartner's own 2025 figure backs her up directly: roughly 40% of enterprise agentic AI projects will be cancelled by 2027.

That statistic is easy to misread. The instinct is to blame the model: pick something smarter, prompt it more carefully, add a few more examples to the system prompt, and the failures will stop. They won't. GPT-5, Claude 4.6, and Gemini 2.5 are all astonishing pieces of engineering. The failures Gartner is counting are not model failures. They are architectural. Teams picked the wrong shape of agent for the problem in front of them, and nobody had handed them the vocabulary to pick correctly in the first place.

This course exists to be that vocabulary.

The two questions nobody asks in a tutorial

Almost every agent tutorial shows exactly one pattern: a chat loop, a tool call, a response. It's a fine way to learn how the plumbing works, and a poor way to learn how to decide what to build, because it never asks the two questions that actually determine an agent's shape.

Who is waiting for this agent to finish?

A human on a live call, watching every turn? A person who'll glance at a Slack message tomorrow morning? Nobody at all, just a log line that a system will page someone about if it's wrong?

Who decides what happens next: the LLM, or our code?

Does the model improvise its way through the task, choosing its own next step at each turn? Or does it slot into a path we designed in advance, where the model only ever supplies language and our code decides the route?

Neither question has a "right" answer in the abstract. They're not a maturity ladder where one end is amateur and the other is professional. They're two independent design choices, and a team that never asks them explicitly ends up making both by accident. That's the failure mode behind Gartner's 40%: not a bad model, but an agent built for one set of answers deployed into a situation that needed the other.

Asked honestly, before any code gets written, these two questions determine the latency budget you can afford, the security model the system needs, what a failure looks like when it happens, how much observability the system requires, and the compliance posture the whole thing has to satisfy. They are also, not coincidentally, orthogonal: a system can be watched-live or unattended independent of whether the LLM or the code holds the wheel. That independence is what turns two yes/no questions into a 2x2 grid, which is where this course spends most of its remaining lessons.

Two independent questions, not one maturity ladder
If you've taken Tools, Memory, and Multi-Agent Systems

Lesson 01 of that course already introduced one half of this picture: the headless-automator-versus-conversational-co-worker distinction, worked through with PyNanoClaw (an always-on WhatsApp-triggered loop) against Voice Rasa Agent (a bounded, STT-to-CALM-to-LLM-to-TTS pipeline). That's the same axis this lesson just raised as "who is waiting for this agent to finish." This course isn't re-deriving it from scratch. Its job is different: showing why that axis matters enough to build a whole decision framework around, and how it composes with the second axis (who decides what happens next) into one 2x2 you can place any agent on. If this is your first course, the axis will be new material either way; if you've taken that lesson, treat this as the point where a concept you already have gets a partner and a frame.

Meet Sarah, and the brief that looks simpler than it is

The rest of this course runs on one running example, so it's worth meeting her now. Sarah is a mid-career software engineer, six years into a B2B SaaS company, comfortable in both TypeScript and Python. She's called the OpenAI API directly, shipped a weekend RAG demo, and used an AI coding assistant daily for over a year. What she has not done is ship a production AI feature complete with evaluation, observability, and an actual answer for what happens when the model gets something wrong. That gap, comfortable with the tools, untested by production, is common enough that it's worth naming as its own category, not a personal shortfall.

Sarah's task this quarter: build a customer-support triage system.

ChannelWhat has to happen
Live chatUnderstand the request, route it, reply or escalate, while someone waits
EmailUnderstand the request, route it to the correct queue
Web formSame intake, batched rather than live

Understand what the customer wants, route the ticket to the correct queue, draft a reply when confident enough to, escalate to a human when not.

Read quickly, that brief sounds like one system. It isn't. By the time this course reaches Sarah's full build, her "one" triage system will turn out to occupy four different quadrants of the 2x2 this lesson just set up. A live chat reply has a different waiting human and a different decision-maker than an overnight email sweep does, even though both are "the same feature" on Sarah's roadmap.

Here's where that stops being abstract. Say Sarah wires the live-chat channel and the overnight email sweep to the same underlying agent, because it's one feature and reusing the logic feels efficient. A customer types "my invoice is wrong" in live chat, and the same code path that handles an overnight email lets the model improvise its way through a few tool calls before replying, because that's what worked fine on email, where nobody was watching in real time. On live chat, that same improvisation costs Sarah a customer while they wait on a decision the model didn't need to be making live at all. The brief never said "one architecture, two channels" was wrong. Sarah's own reuse-mindedness said it.

Her actual job, and the reason her brief is the running example for eleven lessons rather than one, is knowing which piece of the system belongs in which quadrant, not building one architecture and hoping it stretches to cover all four.

Quick check — A team's agent project is being cancelled. The retro concludes the model wasn't smart enough and proposes switching to a newer, more capable model as the fix. What does this lesson's framing suggest is missing from that diagnosis?

Sarah's brief, and the two questions behind it, are where this course starts. The next lesson takes the questions off the whiteboard and puts them to work: the five parts every agent shares (LLM, Planning, Tools, Memory, and the Loop that connects them), regardless of which quadrant it ends up living in.

Continue to Lesson 02

The anatomy every agent shares: the five parts every architecture is built from, before the two axes split them apart.

Have a question about this lesson?

Reply here and it goes straight to Rod. Same as replying to one of his emails.