Skip to content

Test the Environment Your Reviewer Will Actually Run

2026-08-11

Your eval passes in the virtual environment you have been editing for three weeks. That environment holds packages you installed by hand, a cached dataset and an API key exported in your shell. The person who reruns your result has none of them.

A research result is only as good as someone else's ability to reproduce it. That holds for a paper's table, an experiment writeup and a take-home work sample alike. Your dev environment passing is evidence about your dev environment.

When Claude Code changes dependencies, data loading or anything path-sensitive, ask for one check against a fresh clone instead of the working tree. For a Python project managed with uv:

bash
git clone <your-repo-url> /tmp/fresh-check
cd /tmp/fresh-check
uv sync --locked
uv run pytest -q tests/smoke

uv's help text describes --locked as: "Assert that the uv.lock will remain unchanged." If the sync would need to update the lockfile, it stops with an error instead of quietly rewriting it. That is the point: you want to learn that the lockfile drifted, not have it repaired behind you.

Expected result: the smoke test passes, or it fails and names the missing file, package or environment variable. Either outcome tells you something the dev environment could not. Record the commit SHA, the output of uv run python --version, and the exact commands beside the result, so another person can repeat it. The site's own bar is a notebook that runs on Colab's Python 3.12; a clean clone is the local version of the same test.

The Claude Code guide states the rule plainly: "If you can't verify it, don't ship it." What a model call costs, and a deployment that survives carries the same idea from a clean checkout to a running service.

Claude Code documentation.