All sections
Latest
Everything published, newest first.
The log
Entries 61–80 of 103
1 entry
1 entry
Check Which AGENTS.md Files Codex Loaded Before Adding Rules
Before a model can follow a rule, the rule has to be in its context. Chapter 21 treats that context as a budget assembled from parts; for Codex, the…Quick thoughtcodexinstructionsUpdated2 entries
A Dark Factory's First Control: Every Claim Carries Its Own Check
Coding agents are fluent at saying they are done. In the dark factory that builds this site, every claim of progress must name the check that would end it, and a gate refuses the ones that do not. What that control catches, and what it cannot.6 min readdark-software-factoriesAI control
Catch an Agent Citing Papers Its Tools Never Returned
sovereign-agent's research example checks every arXiv ID in the agent's report against what its search tool returned. Read the audit, run it, and measure fabrication rates.7 min readagentsevaluation
1 entry
Compare Cost per Accepted Result, Not per Token or per Session
A cheaper model, or a cheaper Claude Code setup, can cost more per result you keep. A call or a session whose output you throw away cost money and bought…Quick thoughtclaude-codecost1 entry
Keep Your Claude Code Overrides Out of a Shared Research Repo
A research repo shared by four people needs the same permissions and hooks for all of them. It does not need your model override, the path to your scratch…Quick thoughtclaude-codeconfiguration1 entry
Prove a Claude Code Hook Blocks Before You Trust It
Say you write a PreToolUse hook so Claude Code can never edit data/test/, the held-out split your reported numbers depend on.Quick thoughtclaude-codehooks1 entry
Review an MCP Server's Tools Before an Agent Touches Your Runs
You connect an MCP server for your experiment tracker so the agent can read past runs. The same server may also expose tools that tag, overwrite or delete runs.Quick thoughtclaude-codemcp1 entry
Check What Bash(uv run *) Actually Allows Before You Grant It
An allow rule is a threat model in one line. Before you save Bash(uv run ) so Claude Code can run experiments without prompting, write down the worst command…Quick thoughtclaude-codepermissions1 entry
Test the Environment Your Reviewer Will Actually Run
Your eval passes in the virtual environment you have been editing for three weeks. That environment holds packages you installed by hand, a cached dataset and…Quick thoughtclaude-codeverification1 entry
Preview a Mapping Before the Agent Renames Your Run Directories
Renaming 300 run directories into a clean modellrseed scheme is easy to ask for and hard to undo when two runs collapse into one name. lr=1e-3 and lr=0.001…Quick thoughtclaude-codefile-operations1 entry
Give Claude a Failing Eval Fixture Before You Ask for a Fix
"The accuracy looks too high" gives Claude Code nothing it can check. A three-row fixture does. Suppose the eval set was merged from two shards and some…Quick thoughtclaude-codetesting1 entry
What a Dark Factory Must Build Itself When Agents Are Cheap
Every station in a dark software factory is a make-or-buy decision. Coase and Williamson worked out the answer decades before agents: buy what is generic, build what is specific to you, and keep the right to decide when the specification runs out.8 min readdark-software-factoriesmake or buy
7 entries
In a Dark Factory, Checking Is the Real Price of an Answer
As model calls get cheaper, what a dark software factory's output costs comes to be set by review time and acceptance rate, not tokens. The arithmetic, the error bar that decides a route, and the accounting that keeps it honest.8 min readeconomics of AIinference-cost
Stop Being the Agent's Verification Loop
You ask if the change is done. The agent says yes. You rerun the eval, and it isn't. The Claude Code best-practices guide names the position you are in…Quick thoughtclaude-codeverificationTest What a Claude Code Permission Rule Actually Allows, in a Sandbox
A Claude Code permission rule is a string pattern, not an intent. Write the commands it should stop, run them through a headless session in a throwaway VM, and count the unintended allows before you trust the rule.11 min readclaude-codeagent-security
Using a Coding Agent on Research Code: Ask Where the Expected Values Came From
An agent can refactor an eval script and pass every test because it rewrote the expected values. A short guide to using Claude Code or any coding agent on research code and take-homes, built around that check.7 min readclaude-codecoding-agents
What an Agent Loop Costs: Quadratic Resend, Cache Reads and Cost per Solved Task
Every call in an agent loop resends the whole context, so input grows with the square of the calls. Derive the cost with caching, price the solved task with an interval, with Claude Code as the worked instance.10 min readclaude-codeinference-cost
When to Let a Coding Agent Run Unsupervised, Using Public Evidence
Decide how much autonomy a coding agent gets from what the task can be checked against, how cheaply it undoes, and what METR's published studies do and do not show.9 min readclaude-codecoding-agents
Your Chat History Is Not a Lab Notebook: Make Agent Work Resumable
Keep a lab notebook for agent-driven research: the commit, config, data version, results with their error bars and unresolved jobs. Then test whether a fresh session can continue from it alone.7 min readreproducibilityresearch engineering
Continued on the next page.
To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.
