Run /init, Then Test Every Command It Wrote Down
/init drafts a starter CLAUDE.md from the repository; the memory guide says that if one already exists, it suggests improvements instead of overwriting it. Read the draft as two kinds of lines, because each needs different treatment.
Claims describe the repo: "run the tests with uv run pytest -q," "the small eval is scripts/eval.py --subset 200." A claim can be tested, and a generated one can be wrong. Run every command, verbatim, in a fresh shell; each should exit 0. A command that only works with an environment variable you happened to have set belongs in the file with that variable, or out of it. Then keep that run in CI, so the file fails a build when the repo changes under it:
grep -oE '`(uv|make|python)[^`]*`' CLAUDE.md | tr -d '`' > /tmp/documented.txt
while read -r cmd; do sh -c "$cmd" </dev/null || { echo "stale: $cmd"; exit 1; }; done < /tmp/documented.txtRead the extracted list before the first run; it executes whatever the file says.
Rules constrain the agent: "never edit data/processed/ by hand." The memory guide says Claude treats these files as context, not enforced configuration. A rule in context is followed at some rate you have not measured; a test runs every time. Turn each "never" into a check that fails: regenerate data/processed/ with scripts/build_data.py into a temporary directory and fail if it differs from the committed copy. For an action that must happen during a session, the guide points to a hook; this note puts a scorer check in one.
What stays as prose is what no test can check: why the full eval takes two hours, and that you iterate on the 200-case subset and run the full suite before reporting a number. Keep the file under about 200 lines, as the guide suggests, and run /context in a new session to confirm it loaded.
Whether the remaining prose changes behavior is a separate measurement, covered in testing whether a shorter file helps and in the book's skills chapter.
