All sections
Latest
Everything published, newest first.
The log
Entries 121–140 of 167 · by original publication date
4 entries
Build One Station of a Dark Factory: A Typed Tool With a Receipt
Every station in a dark software factory is a tool an agent calls. Wrap a Python function in a typed tool contract, break it three ways, and record a receipt the factory can rerun: commit, versions, command, output and what was not checked.9 min readdark-software-factoriespython
Catch an Agent Citing Papers Its Tools Never Returned
sovereign-agent's research example checks every arXiv ID in the agent's report against what its search tool returned. Read the audit, run it, and measure fabrication rates.7 min readagentsevaluation
Teach Claude That a Blank Eval Score Is Not a Zero
In an eval results table, a blank score can mean the run timed out, the API returned an error or the grader crashed.Quick thoughtclaude-codedata-quality1 entry
Compare Cost per Accepted Result, Not per Token or per Session
A cheaper model, or a cheaper Claude Code setup, can cost more per result you keep. A call or a session whose output you throw away cost money and bought…Quick thoughtclaude-codecost1 entry
Keep Your Claude Code Overrides Out of a Shared Research Repo
A research repo shared by four people needs the same permissions and hooks for all of them. It does not need your model override, the path to your scratch…Quick thoughtclaude-codeconfiguration1 entry
Prove a Claude Code Hook Blocks Before You Trust It
Say you write a PreToolUse hook so Claude Code can never edit data/test/, the held-out split your reported numbers depend on.Quick thoughtclaude-codehooks1 entry
Review an MCP Server's Tools Before an Agent Touches Your Runs
You connect an MCP server for your experiment tracker so the agent can read past runs. The same server may also expose tools that tag, overwrite or delete runs.Quick thoughtclaude-codemcp1 entry
Check What Bash(uv run *) Actually Allows Before You Grant It
An allow rule is a threat model in one line. Before you save Bash(uv run ) so Claude Code can run experiments without prompting, write down the worst command…Quick thoughtclaude-codepermissions1 entry
Test the Environment Your Reviewer Will Actually Run
Your eval passes in the virtual environment you have been editing for three weeks. That environment holds packages you installed by hand, a cached dataset and…Quick thoughtclaude-codeverification1 entry
Trace One Failed Eval Item Through the Whole Harness
Your eval run finished, the score dropped four points, and the log has a hundred warnings. The loudest one is rarely the cause.Quick thoughtclaude-codedebugging1 entry
Preview a Mapping Before the Agent Renames Your Run Directories
Renaming 300 run directories into a clean modellrseed scheme is easy to ask for and hard to undo when two runs collapse into one name. lr=1e-3 and lr=0.001…Quick thoughtclaude-codefile-operations1 entry
Give Claude a Failing Eval Fixture Before You Ask for a Fix
"The accuracy looks too high" gives Claude Code nothing it can check. A three-row fixture does. Suppose the eval set was merged from two shards and some…Quick thoughtclaude-codetesting2 entries
Check That Cleaning an Eval CSV Keeps Its Meaning
An eval results CSV carries meaning a cleanup script can erase. Item ID 00127 is a string, not the number 127.Quick thoughtclaude-codedata-qualityWhat a Dark Factory Must Build Itself When Agents Are Cheap
Every station in a dark software factory is a make-or-buy decision. Coase and Williamson worked out the answer decades before agents: buy what is generic, build what is specific to you, and keep the right to decide when the specification runs out.8 min readdark-software-factoriesmake or buy
5 entries
Clean an Eval Set With a Coding Agent, Then Prove It Dropped Nothing
An eval set merged from three sources has mixed label spellings, near-duplicate prompts and empty references. A coding agent will tidy it. Three counts, computed by your own script, show whether it dropped a row that would fake a gain.8 min readevaluationdata-quality
In a Dark Factory, Checking Is the Real Price of an Answer
As model calls get cheaper, what a dark software factory's output costs comes to be set by review time and acceptance rate, not tokens. The arithmetic, the error bar that decides a route, and the accounting that keeps it honest.8 min readeconomics of AIinference-cost
Stop Being the Agent's Verification Loop
You ask if the change is done. The agent says yes. You rerun the eval, and it isn't. The Claude Code best-practices guide names the position you are in…Quick thoughtclaude-codeverificationTest What a Claude Code Permission Rule Actually Allows, in a Sandbox
A Claude Code permission rule is a string pattern, not an intent. Write the commands it should stop, run them through a headless session in a throwaway VM, and count the unintended allows before you trust the rule.11 min readclaude-codeagent-security
Using a Coding Agent on Research Code: Ask Where the Expected Values Came From
An agent can refactor an eval script and pass every test because it rewrote the expected values. A short guide to using Claude Code or any coding agent on research code and take-homes, built around that check.7 min readclaude-codecoding-agents
Continued on the next page.
To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.
