Latest

Thu, Aug 6, 2026Thursday, August 6, 20263 entries

AnalysisBuild systems around models

What an Agent Loop Costs: Quadratic Resend, Cache Reads and Cost per Solved Task

Every call in an agent loop resends the whole context, so input grows with the square of the calls. Derive the cost with caching, price the solved task with an interval, with Claude Code as the worked instance.10 min readclaude-codeinference-cost

GuideBuild systems around models

When to Let a Coding Agent Run Unsupervised, Using Public Evidence

Decide how much autonomy a coding agent gets from what the task can be checked against, how cheaply it undoes, and what METR's published studies do and do not show.9 min readclaude-codecoding-agents

GuideBuild systems around models

Your Chat History Is Not a Lab Notebook: Make Agent Work Resumable

Keep a lab notebook for agent-driven research: the commit, config, data version, results with their error bars and unresolved jobs. Then test whether a fresh session can continue from it alone.7 min readreproducibilityresearch engineering

Wed, Aug 5, 2026Wednesday, August 5, 20262 entries

EssayBuild systems around models· Food for Thought

A Dark Factory Beyond Software, and the Review Arithmetic That Caps It

A dark software factory turns requests into tested software with no person on each change. The same pattern runs any work you can write as code. What caps it is review, not compute: the arithmetic, the control layers, and where it breaks.9 min readdark-software-factoriesverification

NoteBuild systems around models

Test the Two-Correction Rule Instead of Taking It on Faith

The Claude Code best-practices guide gives a specific threshold: "If you've corrected Claude more than twice on the same issue in one session, the context is…Quick thoughtclaude-codecontext

Tue, Aug 4, 2026Tuesday, August 4, 20261 entry

NoteBuild systems around models

Prune Your CLAUDE.md, Then Test Whether Adherence Moves

Three hundred lines of instructions, and Claude still ignores the rule about where results files go. So you add the rule again in capitals.Quick thoughtclaude-codecontext

Mon, Aug 3, 2026Monday, August 3, 20261 entry

NoteBuild systems around models

Delegate Codebase Discovery, Then Spot-Check the Handoff

Finding where an unfamiliar research repo computes pass@k, or where it filters the eval set before scoring, can take dozens of file reads before you reach the…Quick thoughtclaude-codecontext

Sat, Aug 1, 2026Saturday, August 1, 20261 entry

NoteMeasure models

Let Claude Interview You Before You Build an Eval

"Can you build an eval for tool-use regressions?" sounds like one task. It hides a dozen decisions: which model and snapshot, what counts as a pass, how many…Quick thoughtclaude-codepractice

Fri, Jul 31, 2026Friday, July 31, 20261 entry

NoteBuild systems around models

Put the Check Your Numbers Depend On in a Hook

A line in CLAUDE.md such as "rerun the scorer tests after editing metrics.py" is followed at some rate.Quick thoughtclaude-codeverification

Thu, Jul 30, 2026Thursday, July 30, 20261 entry

NoteBuild systems around models

A Stop Hook Gates the Turn, Up to a Cap

The session reports that the experiment code is done. The smoke test is red. A Stop hook turns that gap into something the session cannot talk past, within a…Quick thoughtclaude-codeverification

Mon, Jul 27, 2026Monday, July 27, 20261 entry

NoteBuild systems around models

The 7x Agent-Team Figure Applies to Plan Mode: Measure Your Own Ratio

The slug of this note says agent teams cost seven times more. The documented claim is narrower, and the difference matters when you plan compute or explain a…Quick thoughtclaude-codeeconomics

Sun, Jul 26, 2026Sunday, July 26, 20261 entry

NoteBuild systems around models

Fan Out a Research-Repo Migration One Bounded Run at a Time

You need to migrate 200 experiment configs to a new schema, or move 200 eval tasks onto a new harness API.Quick thoughtclaude-codepractice

Sat, Jul 25, 2026Saturday, July 25, 20261 entry

NoteBuild systems around models

Give Your Ablation Recipe Arguments, Then Check Each Run

You keep typing the same experiment recipe: train with this config and seed, evaluate on the fixed 200-case subset, write the results with the commit hash.Quick thoughtclaude-codepractice

Fri, Jul 24, 2026Friday, July 24, 20261 entry

NoteBuild systems around models

Pick a Subagent's Model by Measuring It on Your Own Tasks

The slug says run cheap subagents and reserve Opus for the hard part. That is a reasonable hypothesis, not a rule.Quick thoughtclaude-codeeconomics

Thu, Jul 23, 2026Thursday, July 23, 20261 entry

NoteBuild systems around models

Measure What Survives Compaction, Then Write the Rest Down

Three hours into chasing a loss spike, you hold ten facts that matter: the seed that reproduces it, the step where it starts, the config hash, the…Quick thoughtclaude-codecontext

Wed, Jul 22, 2026Wednesday, July 22, 20261 entry

NoteBuild systems around models

Worktrees Separate Files, Not Experiments

You want two agents at once: one refactoring the data loader, one adding an ablation flag. Separate git worktrees give each its own directory and branch, so…Quick thoughtclaude-codepractice

Tue, Jul 21, 2026Tuesday, July 21, 20261 entry

NoteBuild systems around models

Measure What a Second Model Catches Before Adding It to Review

Measuring one reviewer means its recall on planted bugs and its precision on real findings. This note asks a different question: does a second reviewer catch…Quick thoughtclaude-codeverification

Mon, Jul 20, 2026Monday, July 20, 20261 entry

NoteBuild systems around models

Look at the Rendered Figure, Not Just the Exit Code

Claude Code edits the script that draws your results figure. It exits 0. The figure can still be wrong: error bars missing, a legend covering the curve you're…Quick thoughtclaude-codeverification

Sun, Jul 19, 2026Sunday, July 19, 20261 entry

NoteBuild systems around models

Point the Agent at the Exact File, Revision and Failing Case

Your eval reports 0.71 on a benchmark and the paper reports 0.78. You already suspect the answer normalization in eval/metrics.py, and you have one case where…Quick thoughtclaude-codecontext

Sat, Jul 18, 2026Saturday, July 18, 20261 entry

NoteBuild systems around models

Auto Mode Decides; the Sandbox Enforces. Test Which One Stops What

An agent running your eval harness reads untrusted text all day: benchmark prompts, scraped pages, other models' outputs.Quick thoughtclaude-codesecurity

Continued on the next page.

To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.