All sections
Latest
Everything published, newest first.
The log
Entries 21–40 of 55
1 entry
1 entry
Parse Codex's Final Result, Not Its Progress Stream
A sweep driver calls codex exec once per config and builds a results table. If it parses everything the command prints, two things go wrong.Quick thoughtcodexautomation1 entry
A Codex Worktree's Result Depends on Files Git Doesn't Carry
Codex ran your ablation in a separate worktree and reports a two-point gain. Before that number goes in a results table, to a teammate or into the trash with…Quick thoughtcodexworktrees1 entry
Check Which AGENTS.md Files Codex Loaded Before Adding Rules
Before a model can follow a rule, the rule has to be in its context. Chapter 21 treats that context as a budget assembled from parts; for Codex, the…Quick thoughtcodexinstructionsUpdated1 entry
Compare Cost per Accepted Result, Not per Token or per Session
A cheaper model, or a cheaper Claude Code setup, can cost more per result you keep. A call or a session whose output you throw away cost money and bought…Quick thoughtclaude-codecost1 entry
Keep Your Claude Code Overrides Out of a Shared Research Repo
A research repo shared by four people needs the same permissions and hooks for all of them. It does not need your model override, the path to your scratch…Quick thoughtclaude-codeconfiguration1 entry
Prove a Claude Code Hook Blocks Before You Trust It
Say you write a PreToolUse hook so Claude Code can never edit data/test/, the held-out split your reported numbers depend on.Quick thoughtclaude-codehooks1 entry
Review an MCP Server's Tools Before an Agent Touches Your Runs
You connect an MCP server for your experiment tracker so the agent can read past runs. The same server may also expose tools that tag, overwrite or delete runs.Quick thoughtclaude-codemcp1 entry
Check What Bash(uv run *) Actually Allows Before You Grant It
An allow rule is a threat model in one line. Before you save Bash(uv run ) so Claude Code can run experiments without prompting, write down the worst command…Quick thoughtclaude-codepermissions1 entry
Test the Environment Your Reviewer Will Actually Run
Your eval passes in the virtual environment you have been editing for three weeks. That environment holds packages you installed by hand, a cached dataset and…Quick thoughtclaude-codeverification1 entry
Preview a Mapping Before the Agent Renames Your Run Directories
Renaming 300 run directories into a clean modellrseed scheme is easy to ask for and hard to undo when two runs collapse into one name. lr=1e-3 and lr=0.001…Quick thoughtclaude-codefile-operations1 entry
Give Claude a Failing Eval Fixture Before You Ask for a Fix
"The accuracy looks too high" gives Claude Code nothing it can check. A three-row fixture does. Suppose the eval set was merged from two shards and some…Quick thoughtclaude-codetesting1 entry
Stop Being the Agent's Verification Loop
You ask if the change is done. The agent says yes. You rerun the eval, and it isn't. The Claude Code best-practices guide names the position you are in…Quick thoughtclaude-codeverification1 entry
Test the Two-Correction Rule Instead of Taking It on Faith
The Claude Code best-practices guide gives a specific threshold: "If you've corrected Claude more than twice on the same issue in one session, the context is…Quick thoughtclaude-codecontext1 entry
Prune Your CLAUDE.md, Then Test Whether Adherence Moves
Three hundred lines of instructions, and Claude still ignores the rule about where results files go. So you add the rule again in capitals.Quick thoughtclaude-codecontext1 entry
Delegate Codebase Discovery, Then Spot-Check the Handoff
Finding where an unfamiliar research repo computes pass@k, or where it filters the eval set before scoring, can take dozens of file reads before you reach the…Quick thoughtclaude-codecontext1 entry
Put the Check Your Numbers Depend On in a Hook
A line in CLAUDE.md such as "rerun the scorer tests after editing metrics.py" is followed at some rate.Quick thoughtclaude-codeverification1 entry
A Stop Hook Gates the Turn, Up to a Cap
The session reports that the experiment code is done. The smoke test is red. A Stop hook turns that gap into something the session cannot talk past, within a…Quick thoughtclaude-codeverification1 entry
The 7x Agent-Team Figure Applies to Plan Mode: Measure Your Own Ratio
The slug of this note says agent teams cost seven times more. The documented claim is narrower, and the difference matters when you plan compute or explain a…Quick thoughtclaude-codeeconomics1 entry
Fan Out a Research-Repo Migration One Bounded Run at a Time
You need to migrate 200 experiment configs to a new schema, or move 200 eval tasks onto a new harness API.Quick thoughtclaude-codepracticeContinued on the next page.
To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.
