All sections
Latest
Everything published, newest first.
The log
Entries 61–80 of 93
1 entry
1 entry
Prove a Claude Code Hook Blocks Before You Trust It
Say you write a PreToolUse hook so Claude Code can never edit data/test/, the held-out split your reported numbers depend on.Quick thoughtclaude-codehooks1 entry
Review an MCP Server's Tools Before an Agent Touches Your Runs
You connect an MCP server for your experiment tracker so the agent can read past runs. The same server may also expose tools that tag, overwrite or delete runs.Quick thoughtclaude-codemcp1 entry
Check What Bash(uv run *) Actually Allows Before You Grant It
An allow rule is a threat model in one line. Before you save Bash(uv run ) so Claude Code can run experiments without prompting, write down the worst command…Quick thoughtclaude-codepermissions1 entry
Test the Environment Your Reviewer Will Actually Run
Your eval passes in the virtual environment you have been editing for three weeks. That environment holds packages you installed by hand, a cached dataset and…Quick thoughtclaude-codeverification1 entry
Trace One Failed Eval Item Through the Whole Harness
Your eval run finished, the score dropped four points, and the log has a hundred warnings. The loudest one is rarely the cause.Quick thoughtclaude-codedebugging1 entry
Preview a Mapping Before the Agent Renames Your Run Directories
Renaming 300 run directories into a clean modellrseed scheme is easy to ask for and hard to undo when two runs collapse into one name. lr=1e-3 and lr=0.001…Quick thoughtclaude-codefile-operations1 entry
Give Claude a Failing Eval Fixture Before You Ask for a Fix
"The accuracy looks too high" gives Claude Code nothing it can check. A three-row fixture does. Suppose the eval set was merged from two shards and some…Quick thoughtclaude-codetesting1 entry
Check That Cleaning an Eval CSV Keeps Its Meaning
An eval results CSV carries meaning a cleanup script can erase. Item ID 00127 is a string, not the number 127.Quick thoughtclaude-codedata-quality1 entry
Stop Being the Agent's Verification Loop
You ask if the change is done. The agent says yes. You rerun the eval, and it isn't. The Claude Code best-practices guide names the position you are in…Quick thoughtclaude-codeverification1 entry
Test the Two-Correction Rule Instead of Taking It on Faith
The Claude Code best-practices guide gives a specific threshold: "If you've corrected Claude more than twice on the same issue in one session, the context is…Quick thoughtclaude-codecontext1 entry
Prune Your CLAUDE.md, Then Test Whether Adherence Moves
Three hundred lines of instructions, and Claude still ignores the rule about where results files go. So you add the rule again in capitals.Quick thoughtclaude-codecontext1 entry
Delegate Codebase Discovery, Then Spot-Check the Handoff
Finding where an unfamiliar research repo computes pass@k, or where it filters the eval set before scoring, can take dozens of file reads before you reach the…Quick thoughtclaude-codecontext1 entry
Let Claude Interview You Before You Build an Eval
"Can you build an eval for tool-use regressions?" sounds like one task. It hides a dozen decisions: which model and snapshot, what counts as a pass, how many…Quick thoughtclaude-codepractice1 entry
Put the Check Your Numbers Depend On in a Hook
A line in CLAUDE.md such as "rerun the scorer tests after editing metrics.py" is followed at some rate.Quick thoughtclaude-codeverification1 entry
A Stop Hook Gates the Turn, Up to a Cap
The session reports that the experiment code is done. The smoke test is red. A Stop hook turns that gap into something the session cannot talk past, within a…Quick thoughtclaude-codeverification1 entry
The 7x Agent-Team Figure Applies to Plan Mode: Measure Your Own Ratio
The slug of this note says agent teams cost seven times more. The documented claim is narrower, and the difference matters when you plan compute or explain a…Quick thoughtclaude-codeeconomics1 entry
Fan Out a Research-Repo Migration One Bounded Run at a Time
You need to migrate 200 experiment configs to a new schema, or move 200 eval tasks onto a new harness API.Quick thoughtclaude-codepractice1 entry
Give Your Ablation Recipe Arguments, Then Check Each Run
You keep typing the same experiment recipe: train with this config and seed, evaluate on the fixed 200-case subset, write the results with the commit hash.Quick thoughtclaude-codepractice1 entry
Pick a Subagent's Model by Measuring It on Your Own Tasks
The slug says run cheap subagents and reserve Opus for the hard part. That is a reasonable hypothesis, not a rule.Quick thoughtclaude-codeeconomicsContinued on the next page.
To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.
