Eight Real Agent Failures, Explained

AttributeError: 'ToolRegistry' object has no attribute 'list_tools'. Demo code called tools.list_tools(); the registry never had that method. The offline tests never exercised this path, because the offline demos don't print the tool list. Fixed with a defensive fallback: try the call, fall back to sorted(tools._tools.keys()) on AttributeError. The lesson: demo code that runs at the top of a user-facing script is a common blind spot for a test suite that only exercises the library underneath it.

Output showed the default models (Qwen3-Next-80B-A3B-Thinking, Qwen/Qwen3-32B) instead of the ones set in .env (MiniMaxAI/MiniMax-M2.5, Qwen/Qwen3-235B-A22B-Instruct-2507). Config.from_env() reads SOVEREIGN_AGENT_LLM_PLANNER_MODEL, not the short name PLANNER used in the .env file. Config prefixes exist to prevent collisions between unrelated tools sharing an environment — but that safety trades away discoverability. The fix isn't a code change, it's documenting the full variable names in .env.example, not the aspirational short ones.

make example-research-real finished, and ls sessions/ returned "No such file or directory." The demo code used tempfile.TemporaryDirectory(), which deletes itself the moment the Python process exits — taking every artifact from the run with it. Fixed by persisting real runs under ./sessions/ and reserving the tempdir for offline/scripted mode only. If you can't inspect what happened after a run, you can't debug anything. Tests can afford to evaporate their fixtures. Demos cannot.

The research assistant ran and produced: outcome: handoff_to_structured, (No report written.). The planner had split the task into three subgoals and assigned two of them to a structured half — except the research scenario has no structured half at all. The handoff went into the void. The planner's prompt lists "loop or structured" as options regardless of what's actually wired into the scenario; it doesn't check. Fixed with explicit task constraints: "This entire task stays in the LOOP half. Do NOT assign subgoals to the structured half." First instance of a pattern that recurs through this whole list — the model reaching for a capability the scenario doesn't need, because nothing told it the capability wasn't available.

Source code was inlined directly in the task prompt for the code-reviewer scenario. The trace showed read_file(path="workspace/source.py") → not found, then list_files(path="workspace") → empty, then complete_task(status="failed", reason="No source file found"). The model never called the analyzer tool at all — it saw the word "file" in a tool name, ignored the inlined source sitting right there in its own context, and went hunting the disk instead. The first fix attempt strengthened the prompt with explicit "do NOT try to read it from the workspace" language. The model called list_files twice anyway. When a directory-listing tool exists, models reflexively reach for it to "orient themselves" — training data is full of trajectories that start that way, and no prompt reliably overrides the reflex. The fix that actually worked: don't offer the tool. Narrow the registry to exactly what the scenario needs, same move as Decision 8 from the previous lesson.

With the narrowed registry, the scenario ran clean: output written, complete_task called, every ticket green, every manifest verified. The review itself described functions named add, multiply, and divide, all "implemented correctly," zero issues found. The actual source file has no such functions — it has one function called do_everything_in_one_function, with wildcard imports and too many print statements. Root cause, found by logging exactly what the analysis tool received: a _TOOL_CALL_LOG capturing source_matches_sample == False on the first 80 characters, proving the LLM had fabricated 142 bytes of plausible-looking Python, run it through the analyzer, gotten 2 findings back on that fiction, and blended those findings with its own training-data knowledge into a fluent, wrong review. The fix: stop inlining source in prompts entirely. Seed it to workspace/sample.py on disk before the run, and give the tool a filename to read itself — no copying, no re-tokenization, no compression-induced drift between what the file says and what the model repeats.

With the file-on-disk pattern, the tool definitely read the real source. But nothing forced the LLM to actually use what the tool returned — it could still ignore four real findings and write a report describing four different, fabricated ones, and nothing in the framework's tickets or manifests would catch it. This is the gap between structural correctness (the tool was called, the manifest verified, the state advanced) and semantic correctness (the output of tool N was meaningfully used as input to tool N+1). Tickets guarantee the first. They say nothing about the second — which is exactly the gap this whole arc opened on in Lesson 27.

The final run: a correct, real code review, all four real findings present, dataflow verified clean. And still: ⚠ ipc/session_complete.json missing. The LLM wrote the review, then emitted a plain-text summary of what it had done instead of calling the formal complete_task sentinel. The session ended cleanly in practice, but the completion record was never written. Models sometimes skip protocol steps once the "real work" feels finished to them. The fix isn't a prompt arms race — it's accepting the behavior, treating a missing sentinel as informational rather than fatal, and considering a framework-level auto-close for the next version.