The Harness at Work: Verification, MCP, and a New Trade Learned Live
Seven keystrokes, /reconcile 2026-07, are about to fire a full reconciliation procedure against Xolo Consulting's books, and this lesson watches it happen twice: once to show what actually loads into the model's context when a shortcut like that runs, and once more when an independent check recomputes the resulting report from raw data and delivers a real verdict, confirmed or refuted, never a hedge. Behind that one command sit five primitives from the previous unit, in the class's own working metaphor: a handbook, procedures, buttons, staff, and reflexes, none of which have run yet at the point this lesson opens. By the end, the organization learns an entirely new trade live, in front of you, closing the scoreboard that's been open since the start of the class.
This is the fourth unit of Class 3. It assumes the five primitives from the previous unit, memory, skills, commands, subagents, and hooks, and puts every one of them to work on Xolo Consulting for real.
Seven keystrokes, then a second opinion
/clear first, to start clean. Before typing anything else, /context shows the window's contents: system prompt, CLAUDE.md, tool definitions, nothing from the invoice-reconciliation skill anywhere in it, because nothing has asked for that skill yet.
Then, seven keystrokes: /reconcile 2026-07.
The command file injects its own template, which names the skill, and the skill's description matches the task well enough that its full procedure loads into context. /context run again, mid-plan, shows the difference directly: the skill's body is now present in the window, where a moment ago it wasn't. That's progressive disclosure, not as an abstract claim about how the system works, but as two screenshots of the same command, one before and one after, with a visible difference between them.
Worth being precise about what actually happened there, because it's easy to wave past. Before the command ran, the skill cost nothing at all, not a single token of the window, because only its name and description were ever candidates for loading, and neither had matched anything yet. The moment the task matched, the full procedure, every numbered step of it, joined the context in one movement. That's the entire economic argument for skills over one giant CLAUDE.md: fifty procedures can sit in a repository at effectively zero standing cost, and only the one that's actually relevant to today's task ever gets paid for.
It didn't disappear. It decomposed. The rules that used to sit inline in a hand-typed brief now live in CLAUDE.md, always loaded, never retyped. The step-by-step procedure that used to get spelled out fresh every time now lives in the skill, loaded only when a task like this one comes along. The entry point that used to be eight or ten lines of typed instruction now lives in a seven-keystroke command. Nothing about the actual work got smaller. It just stopped needing to be rebuilt from memory every single time someone needed it.
The plan comes back, gets approved, and executes the same way the properly briefed run did in the earlier lesson, permission prompts narrating each consequential step as it happens. A report lands in reports/.
Do we trust it? Wrong question
Here's the moment worth pausing on. The report says it's done. A naive reader, or an earlier version of you, an hour or two into this course, would simply believe it. But "do we trust it" is the wrong question to be asking at all, because trust is a feeling, and feelings are exactly what the RCT evidence from Class 1's material showed can mis-sign an effect's direction, not just its size. The right question is narrower and answerable: can we verify it?
Asking the verifier subagent to check reports/2026-07-reconciliation.md spawns it through the Task tool, into its own separate context window, with no visibility into how the report was produced, only the raw data and the report itself. It recomputes every headline number independently, from data/, not from reading the report's own reasoning first, and returns one of exactly two verdicts: confirmed, or refuted, never something softer like "mostly fine."
:::checkpoint{id="itam-w03s4-verifier-verdict" question="The verifier subagent is asked to check a reconciliation report. Why does its job description forbid a verdict like "mostly fine" or "looks reasonable"?" options="Because those verdicts take longer to compute,Because a soft verdict lets an unverified number slip through as if it had been checked,Because the model cannot generate that kind of language,Because "mostly fine" is reserved for a different subagent" answer="2"} A verdict like "mostly fine" sounds like verification without actually forcing every number to be independently recomputed and checked. Confirmed or refuted are the only two outcomes that force a real conclusion: either the verifier's own recomputation matches the report's numbers, or it doesn't. Anything softer creates room for an unverified claim to pass as if it had been checked, which defeats the entire purpose of having an adversarial second context in the first place. :::
If the verdict comes back confirmed, the scoreboard for this run is five of seven, the same five as before, but now independently checked rather than merely evidenced by the same run that produced them. If it comes back refuted, that's not a failure of the exercise, it's the exercise working exactly as designed: the checker caught something the doer missed, and it caught it because the two ran in two entirely separate contexts, not because either one tried harder.
What goes where: the full consolidation
Two tables cover the entire architecture of this class, together. The first covers the primitives that advise a model that is, underneath everything, still a stochastic system.
| Knowledge | Home | Why there |
|---|---|---|
| Always-true rules, project map | CLAUDE.md | Paid for in every session; constitution-grade only |
| Procedure for a recurring task | Skill | Loaded only on match; detail lives at no ongoing cost |
| Human-triggered entry point with arguments | Command | A button; pairs naturally with a skill underneath it |
| A role that needs isolation | Subagent | Its own window, its own tools; no contamination either way |
The second table covers the two homes that don't advise the model at all, they constrain the harness around it, and they're a different species entirely from everything in the first table.
| Something that must happen | Home | Why |
|---|---|---|
| An action that must always occur | Hook | Deterministic; runs regardless of what the model concludes |
| What the agent may touch at all | settings.json | Allow, deny, and everything else asks; the permission perimeter |
Never entrust a guarantee to a probability. Ask of every rule you're about to write down: must this always hold? The pink-table rows. Or does it hold usually, with room for judgment on the specific case? The first table. That single question routes every rule you'll ever write for a system built this way, for the rest of this course and past it.
MCP: extending the tools term
One more piece completes the picture before the final demo, and it's worth understanding properly even though the actual wiring waits for a later week. MCP, the Model Context Protocol, is an open standard for giving an agent new tools. A server exposes a set of capabilities, query this database, post to this channel, read this inbox, and Claude Code discovers and calls them exactly the way it calls any built-in tool.
In equation terms, MCP only extends term three, tools. Nothing else about how the loop works, how permissions gate action, how memory loads, changes because an MCP server is connected. That's precisely why it could wait until this point in the class rather than needing to be introduced on day one.
claude mcp add pg -- npx @modelcontextprotocol/server-postgres
Project-shared servers live in a committed .mcp.json file, and /mcp shows the status of whatever's currently connected. In a few weeks, Xolo gains real plumbing through exactly this mechanism: a real database standing in for the CSVs, email drafts instead of a chat-only summary, a way to post directly to a team channel.
Two costs come with every server worth naming honestly, not just the gains. Every server's tool list occupies real context space the moment it connects, checkable with /context the same way a skill's body is checkable, and a project with a dozen servers wired in can find a meaningful share of its window spent on tool definitions before a single message has been typed. And every server is trust surface: its tools act with your permissions, reading what you can read, writing what you can write, which means curating which MCP servers get added to a project is closer to hiring than to installing a browser extension. The perimeter and the hooks from the previous lesson still govern everything a connected server is allowed to do; MCP extends the tool belt, it doesn't bypass the gate sitting in front of it.
Demo four: the organization learns a trade, live
Two contract traps have sat untouched since the start of the class, the self-contradicting amendment and the unreadable scan. Neither gets hunted down by hand. Instead, Claude Code is asked to write an entirely new skill for itself.
The brief, given in plan mode: create .claude/skills/contract-review/SKILL.md, given the contracts folder, extract per contract the parties, term, end date, whether it auto-renews, the notice-period in days, the value, and a risk flag, high if renewal or notice falls within a hundred and twenty days. The rules matter as much as the extraction: cite the file and section for every value pulled out, and anything contradictory or unreadable goes into an exceptions section with the actual reason, never a guess dressed up as data. Model the whole structure on the invoice-reconciliation skill already sitting in the repository.
It writes the file. Reviewing it happens the same way any pull request gets reviewed, reading it on screen, checking the description field specifically: would this actually trigger on a real contract-renewal question, or is it too vague to fire when it's needed? You are, at this point, code-reviewing a standard operating procedure the agent wrote for itself, the same review discipline from the previous lesson's skills section applied to a skill that didn't exist ten minutes earlier.
/clear. Total amnesia. The conversation that just wrote the new skill is gone entirely, nothing about it survives in context.
"Review our contracts for renewal risk." Eight words, no reference at all to the skill that was just built, no reminder that it exists.
The skill's description matches the task, loads on its own, and the procedure runs.
If the new skill still fires correctly after /clear wipes every trace of the conversation that created it, the knowledge it represents is organizational, not conversational. It didn't survive because someone remembered to mention it. It survived because it's written down in a file that loads on its own, the exact same mechanism that's carried this entire lesson since the moment CLAUDE.md was first read into a fresh session two lessons ago.
The report comes back naming both remaining traps by name. Two contracts get tracked cleanly, dates and terms cited to the file and section they came from. The self-contradicting amendment gets flagged as a conflict for a human to resolve, not silently resolved by picking one of its two contradictory numbers and moving on. The unreadable scan gets marked as needing a human to read it, not guessed at from context clues the way a less disciplined procedure might have been tempted to.
The scoreboard closes
Seven of seven. Zero of them found by anyone pointing at a spreadsheet cell or a contract clause and saying "check this one." All seven found by structure: a constitution that killed trap zero before the first properly briefed run even started, two procedures that turned a one-time good brief into something that fires the same way every time, an adversarial subagent that confirmed rather than merely trusted, and a skill the organization wrote for itself in the time it takes to review a short pull request.
What's next
One lesson remains for the class: the five-step workflow this entire session has actually been demonstrating without naming it directly, the steering habits worth carrying past this course, and the homework that asks you to reproduce a version of this same class on your own machine, against your own case study business, before the next class.
The workflow cycle, the five steering habits, and the homework due before Class 4.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
