
Catch an Agent Citing Papers Its Tools Never Returned
sovereign-agent's research example checks every arXiv ID in the agent's report against what its search tool returned. Read the audit, run it, and measure fabrication rates.

Worked examples and arguments about AI systems: how they work, implementation choices, evaluation, costs, failures and what they change about organizations.
Build a workflow around your coding agent. The dark software factory workshop connects the explanations to one runnable application, acceptance checks and a release decision.
Archive
70 articles, newest first.
1–3 of 3 articles in Practice
Clear filters
sovereign-agent's research example checks every arXiv ID in the agent's report against what its search tool returned. Read the audit, run it, and measure fabrication rates.

An eval set merged from three sources has mixed label spellings, near-duplicate prompts and empty references. A coding agent will tidy it. Three counts, computed by your own script, show whether it dropped a row that would fake a gain.

An agent can refactor an eval script and pass every test because it rewrote the expected values. A short guide to using Claude Code or any coding agent on research code and take-homes, built around that check.
Learn
Save your progress with a free account.
Checking sign-in…
Save your progress with a free account.
Pick up where you left off.
Continue learning at my deskCookie duty
The one he did not eat is a measurement cookie. Say yes and the site loads Google Analytics (GA4), which stores an identifier in your browser and lets us count which lessons get read.
Change your mind any time with Privacy choices in the footer. Read the Privacy Policy
Optional · newsletter
That is saved and settled. Separately, and this is marketing rather than measurement: Rod writes a weekly email — one lesson, one field note — sent through Kit. It has nothing to do with cookies, and skipping it costs you nothing.
We will send a confirmation email. Click the link in it and the next lesson finds you.