Pin the Environment Before You Try to Reproduce a Result

Two people can run the same eval script on different library versions and argue about a number neither can reproduce. Reproducing results is the daily work of a research engineer, and a work sample that says "I pinned the environment first" reads differently from one that says "it works on my machine."

Ask Claude Code to read the manifest and the lockfile before it touches the code, then reproduce the issue in the locked environment. With uv, two commands do most of the work:

uv lock --check        # fails if uv.lock is out of date with pyproject.toml
uv run --locked pytest tests/test_metrics.py   # errors instead of changing uv.lock

Then record what actually ran. Write the versions of the libraries that matter into the result file, next to the score, so a later reader does not have to trust your memory. Do not let the agent install a newer package because an old example used a similar name.

If an upgrade is the proposed fix, put the lockfile change in its own commit and rerun the same eval subset before and after it. If the number moves with the upgrade alone, you have found a dependency change, not a code bug, and the write-up should say so.

The agent's own configuration is part of the environment too. The settings documentation describes a shared .claude/settings.json that you commit so everyone who clones the repository gets the same permissions, hooks and environment variables, and a .claude/settings.local.json for personal overrides that stays out of Git. Run /status in a session to see which settings files it loaded.

The check: on a fresh clone, uv lock --check passes, the reproduction command gives the recorded number, and the result file names the versions that produced it.

Claude Code documentation.