Find the Change You Never Asked the Agent to Make
You asked an agent to fix an axis label on a results plot. The diff also switched the F1 score from macro to micro averaging, because the agent read it as a cleanup. On an imbalanced label set, those two averages can differ a lot, and every number in the draft just moved without anyone deciding it should.
Two checks catch this before it reaches a results table:
git diff --stat main...HEAD # does the file list match the request?
# before.csv: the same command, run on main before the change
uv run python -m eval.run --subset fixed-200 --out after.csv # your repo's eval command
diff before.csv after.csv # a cosmetic change must print nothingA label fix should touch plotting code, not metrics.py. If the results file changes at all, the change was not cosmetic, whatever the commit message says.
Claude Code's best practices describe an adversarial review step: before treating a task as done, have a subagent review the diff in a fresh context against your plan, including a check that nothing outside the task's scope changed. The same page warns that a reviewer asked to find gaps will usually report some even when the work is sound, so tell it to flag only gaps that affect correctness or the stated requirements. The bundled /code-review skill reviews the current diff for correctness bugs in a fresh subagent, and measuring what it catches is a separate job. A bug hunt can pass the F1 switch, because micro averaging is correct code. What is wrong is that nobody asked for it, and only a scope check sees that.
If the unrequested change is a real improvement, keep it, but in its own commit with its own before-and-after numbers and a sentence explaining why. A reviewer can then accept the label fix in a minute and think carefully about the metric change on its own. The book's evaluation chapter covers how to tell whether a moved number is a real change or noise.
The check: the file list matches the request, and for any change described as behavior-preserving, the eval diff is empty.
