Catch an Agent Citing Papers Its Tools Never Returned

An agent searched for papers on retrieval-augmented generation, got nothing back four times, and then wrote a 2,500-character summary of RAG papers anyway. Every step completed. The tools ran, the file was written, the task was marked done. The report was invented.

That run is recorded in a comment in sovereign-agent's research example, next to the check written to catch it:

# Apr 23 2026 trace: 4 empty queries, then a 2500-char
# fabricated summary of RAG papers from training data.

This piece reads that check, shows how to run it, turns it into a measurement with an interval, and points to where the current release and the book take the same idea further.

The audit: a set difference

The example lives in examples/research_assistant/run.py at the repository's v0.2.0 tag. An agent is asked to research a topic using only a web_lookup tool and to write workspace/report.md, citing "each paper's arXiv ID verbatim". The tool is scripted: it returns abstracts from a small fixture, and every call is appended to a log.

After the run, the audit compares two sets. These lines are from the file:

real_arxiv_ids = {aid for c in _TOOL_CALL_LOG for aid in c["arxiv_ids"]}
cited_ids = set(re.findall(r"\b(\d{4}\.\d{4,5})\b", report_text))
fabricated_ids = cited_ids - real_arxiv_ids

Any ID in the report that no tool call returned is printed as fabricated. The function's docstring lists the three failure modes it catches:

1. the report cites an arXiv ID no lookup returned;
2. the report cites a paper title no lookup returned, treated as a warning because titles can be paraphrased;
3. no lookup was called at all, yet a report exists.

A fourth guard catches the April 23 run: when every lookup returned zero results and the report is longer than 500 characters, the audit prints that "no web_lookup call returned results, but a report was still written."

View full-size diagram

The audit only prints; it does not raise. Its docstring says why: "The signal matters more than the failure; students and reviewers need to see which runs fabricated."

What the audit does not prove

A set difference is a narrow check, and it is worth being exact about its reach.

• A returned ID can still be misused: the report can cite a real, returned paper for a claim that paper does not make. The audit sees the ID, not the claim. Checking what a citation actually proves is a separate step.
• The pattern matches only post-2007 arXiv IDs: an older identifier, a DOI or a bare title passes the strict check untouched.
• Titles are a soft check: the audit counts how many returned titles appear in the report, but a paraphrase can miss.

None of that makes the check weak for its purpose. It turns one specific failure, citing a source that was never retrieved, from something a reader might notice into something a script always reports.

Run it

The research example needs the source tree, not the published package: the v0.2.0 pyproject.toml excludes examples* from the wheel. That release declares requires-python = ">=3.12", the Python version Colab runs. The example's own docstring gives the commands; the offline run uses a scripted model and needs no API key:

git clone --branch v0.2.0 --depth 1 https://github.com/profrodai/sovereign-agent sovereign-agent-v0.2.0
cd sovereign-agent-v0.2.0
pip install -e .
python -m examples.research_assistant.run

The live version sends the same task to a real model through an OpenAI-compatible client. At v0.2.0 the default provider is Nebius, and .env.example shows how to point it at another endpoint:

NEBIUS_KEY=... python -m examples.research_assistant.run --real

A live run keeps its session directory, whose layout is defined in sovereign_agent/session/directory.py:

workspace/report.md is the file the audit reads, and logs/trace.jsonl is the append-only trace of what the agent did. These commands follow the repository's files at that tag; this article has not rerun them on September 30, 2026.

Turn the audit into a measurement

One clean run shows the check works. It does not tell you how often a given model fabricates. That is a rate, and a rate needs a sample and an interval. This site has not run the measurement below yet.

The interval matters most when the count is small. If 0 of 20 runs are flagged, the true rate is not shown to be zero. The rule of three puts the 95% upper bound near 3 / n = 3 / 20 = 0.15, and the Wilson interval gives about 0.16. You would need about 300 clean runs to bring that bound down to 1%. Chapter 16 of the book derives the Wilson interval and the sample-size arithmetic.

Where the current release takes the idea

The v0.x framework has since been replaced. The current README says, "The 1.x API intentionally replaces the v0.7 fleet framework", and the 1.x line (v1.4.0, released September 5, 2026) is a smaller agent with one runtime dependency, Pydantic. The research example is not in it. Its principle is, and it moved from an after-the-fact audit into the structure. The README describes an agent that "reports its day from receipts rather than from what the model said", and the tests pin that down:

• test_unknown_effect_and_delivery_cannot_disappear_in_fluent_result stores a model result that says "All complete. We spent nothing." and asserts that the phrase never reaches the operating report, which states the amount from the spending ledger instead.
• test_cli_prints_ledger_report_without_a_model_call asserts that the report command makes no model call at all.
• test_fabricated_evidence_row_is_refused_by_the_database shows that a made-up evidence row cannot even be inserted: the foreign key refuses it.

You can try the current release without installing anything permanent, using the README's own commands:

uvx sovereign-agent@latest doctor
uvx sovereign-agent@latest demo store --mode simulated --root /tmp/first-shift

Build it from scratch

The repository is the finished agent; the book is the path to building it yourself. Each chapter measures a mechanism on a real model before building it, and its notebooks run on Colab. The chapters closest to this piece:

• Chapter 3, the agent loop: the model-and-tool loop the research example runs;
• Chapter 5, memory and retrieval: what the model sees, and why retrieval decides it;
• Chapter 16, evaluation as measurement: error bars and paired comparisons;
• Chapter 20, honest reports: a whole day, reported from records rather than from the model's account.

The exercises and worked solutions are in the book's course repository. The Tools, Memory, and Multi-Agent Systems course tells the story of the earlier design, including the failures that shaped it.

Why this matters in a lab

Grounding is an evaluation problem at every frontier lab. OpenAI's frontier-evals posting lists "factuality" among the behaviors its team trains and measures, and Google DeepMind's agent-quality posting asks for diagnostic tooling "to identify root-cause agent failure modes (e.g., passivity, hallucination, or brittle tool execution)." An interviewer will not ask whether your agent can write a report. They will ask how you know its citations came from its evidence, and how often it fails when the evidence is empty. The answer is a check that needs no model and a rate with an interval.