Sarah's Triage System: One Brief, Four Quadrants

Go back to Sarah's brief from Lesson 01: build a customer-support triage system. Tickets come in, get routed, some get auto-replies, some escalate. Read once, that sounds like a single build — pick an architecture, write the code, ship it. It isn't, and the gap between how the brief reads and what it actually requires is where most of Gartner's 40% comes from.

Before the decomposition, try it yourself. Look at that one-sentence brief again and count how many distinct pieces you see in it — not features, pieces that would each need their own answer to "who's waiting" and "who decides." Hold that number; the brief invites a single question ("is this a chatbot or an agent?") that has no good answer here, because it isn't describing one system. It's describing four, stacked under one product name. Sarah's actual job — the job that makes her an AI Product Engineer rather than someone who picked a framework off a blog post — is noticing that before writing a line of code, then placing each piece separately. Here's the full decomposition; we'll take each layer in turn.

Four rows, four different answers to "who waits" already. That's before we even get to "who decides." One brief. Four quadrants.

Layer A — the live chat widget

Who's waiting? The customer, mid-conversation. Latency budget: under three seconds per turn. Who decides? Not the model. "Is this an enterprise customer with a blocking issue?" is a question with a determinable right answer, and Sarah cannot afford to let an LLM improvise it turn to turn.

That combination (a live human waiting, a decision that has to be right and repeatable) lands Layer A in Conversational + Deterministic. Lesson 04 already named the reference architecture for this quadrant: Rasa CALM, where the LLM interprets what the customer means and a predefined Flow decides what happens next. Sarah's first flow for this layer:

flows:
  handle_incoming_ticket:
    steps:
      - collect: issue_category
      - collect: urgency
      - action: action_lookup_customer
      - action: action_check_known_issues
      - link: escalate_to_human  # if unresolved

The LLM's whole job here is turning "my bill seems weird" into issue_category=billing, urgency=non_blocking. The Flow's job is deciding what to do with that information. If this layer misbehaves, Sarah loses a customer on the spot — which is exactly why determinism in Layer A isn't a stylistic preference, it's the requirement the latency budget and the stakes both impose.

Layer B — the nightly extraction pipeline

Who's waiting? Nobody. This runs at 3 AM. Who decides? The pipeline, which is fixed: pull yesterday's tickets, extract structured fields, validate, upsert to the analytics warehouse, the same four steps every night.

The LLM appears once, doing one job: text in, JSON out, against a fixed schema. It is a step, not the conductor. A failed validation flags the row for human review instead of the pipeline guessing; a failed LLM call retries instead of improvising a substitute.

Headless, nobody waiting, path fully pre-defined: Headless + Deterministic, the quadrant most teams skip because it "isn't agenty enough," and the quadrant where most of the actual production value in a system like Sarah's lives. The boring quadrant pays for the interesting ones.

Layer C — the overnight research agent

Who's waiting? Nobody, until Sarah opens her laptop the next morning. Who decides? The model, because the task can't be pre-defined. "Figure out what's going on with this unresolved ticket" sends different tickets down different paths — different docs, different tools, different reasoning chains — and writing a Flow that anticipates all of them isn't possible the way it was for Layer A.

# Sarah's research loop (pseudocode)
for ticket in unresolved_tickets:
    agent.run(
        goal=f"Research ticket {ticket.id} and draft a reply",
        tools=[search_docs, search_past_tickets, read_codebase],
        max_turns=15,
        budget_usd=0.50,
    )

Notice what's not missing from that pseudocode: max_turns=15 and budget_usd=0.50 are named, concrete constraints on an open-ended task, not an unbounded loop with a hopeful prompt. That's Layer C's real discipline — Headless + Stochastic doesn't mean unsupervised, it means the path is open but the blast radius is bounded. The agent drafts a reply into a review queue, not to the customer directly, and Sarah approves every one before it sends. Lesson 04 already named the reference architecture for this quadrant — a Claw-family, OpenClaw-style agent — so we won't re-derive how those work here; the only new fact is that Sarah's version caps itself at 15 turns and fifty cents per ticket.

Layer D — Sarah, building the system

Who's waiting? Sarah, right now, at her keyboard. Who decides? The model, again — she's working with Claude Code in Cursor, typing things like "refactor this CALM flow to handle account tier lookup," and the agent reads the file, plans the change, and edits multiple files while she reviews and iterates.

That's Conversational + Stochastic: the same "who decides" answer as Layer C, but with a live human on the other end instead of an overnight queue. It's also the quadrant where most engineers already spend their day in 2026, without ever framing it as "operating an agent." The same risk applies: the agent can touch any file, run any shell command, call any MCP server. The only thing standing between Sarah and a bad outcome is that she's in the loop on every meaningful action: a design property, not a personality trait, and the same property Layer C leans on for its own review-queue step.

The four-question method, applied per layer

Here's the actual mechanism behind everything above — not one architectural decision for Sarah's whole system, but the same four questions, asked separately, for each layer:

Run those four questions against any one layer and you get one quadrant — Layer A lands Conversational + Deterministic, Layer B lands Headless + Deterministic, Layer C lands Headless + Stochastic, Layer D lands Conversational + Stochastic. Run them against Sarah's brief as a whole, once, and you get nowhere, because the brief doesn't have one answer to "who's waiting." It has four.

If Sarah had built her entire system in one quadrant instead, she'd have failed a different way depending on which one she picked. All-stochastic, and the compliance team shuts down Layer A before it ships. All-deterministic, and Layer C can't handle the long tail of tickets nobody anticipated. All-conversational, and Layer B's nightly pipeline has no reason to exist. All-headless, and the customer in Layer A has nobody to talk to.

That's the reframe worth carrying forward: answer the two questions first, per layer, and pick the framework second. Teams that reverse the order — pick a framework, then discover which parts of the brief it can't serve — are the ones feeding Gartner's 40% cancellation figure Lesson 01 opened with. Sarah's brief didn't dodge that outcome by being simple. It dodged it by getting decomposed correctly before anyone wrote a Flow or a prompt.