Make Your First Change With Codex

Dollar amounts on this page are in USD. The bill amounts in the exercise are made-up teaching figures, not real charges.

On a research team, much of the code a coding agent touches computes numbers someone will act on: an eval script, a metric, a data loader, the function that fills a results table. A crash is easy to spot. A change that shifts a metric by two points and still passes its tests is not, and it spreads to every row that uses it. So every agent-made change gets the same three questions: what did the code produce before, what does it produce now, and does the difference match what you asked for?

This lesson trains that check. You record a baseline, ask Codex for one bounded change, read the diff, and rerun the baseline yourself. Codex will tell you it is done. Your job is to have your own evidence.

You practice on a stand-in: a small Python bill splitter whose every output you can predict by hand. The version that matters is near the end, the same steps on an eval script that reads frozen model outputs. That version is written out but has not been run yet; the stand-in has.

If you are asked in an interview how you use coding agents, this is evidence you can show: a committed baseline, the diff you read, and the comparison you ran yourself.

What you need

• A terminal, Git, and Python 3.9 or newer. The practice repository uses only Python's standard library.
• The Codex CLI and a way to sign in. OpenAI documents two: "Sign in with ChatGPT for subscription access" and "Sign in with an API key for usage-based access."

On cost, here is what OpenAI's pages said on September 21, 2026. The Codex pricing page lists ChatGPT Plus at "$20" per month and describes it as including "Codex on the web, in the CLI, in the IDE extension, and on iOS." If you sign in with an API key instead, "OpenAI bills API key usage through your OpenAI Platform account at standard API rates." This lesson is one short session. I did not measure what it costs on either route, so check your own usage afterward with /status inside Codex.

Get the practice repository

Download tip-splitter.zip, unzip it, and open a terminal in the tip-splitter folder. It splits a restaurant bill evenly and keeps every amount in whole cents.

It stands in for research code. It is small enough that you can predict every output before the agent touches it, so the before-and-after comparison has only one possible reading. A tiny eval repository with frozen model outputs would make the practice match the real use; this course does not supply one yet, so the eval version near the end is written out, not run.

OpenAI's documentation says Codex starts read-only in a folder that is not under version control, and you need Git for the comparison later. Make the folder a repository and commit the starting point:

git init
git add .
git commit -m "Starting point"

Install Codex and sign in

OpenAI documents several installers. The npm package works the same way on all three platforms:

npm install -g @openai/codex

npm install -g @openai/codex

npm install -g @openai/codex

The documentation also lists a standalone installer and brew install --cask codex. Whichever you use, confirm the version and sign in:

codex --version
codex login

codex login opens the browser sign-in. This course was checked against codex-cli 0.155.1. If yours is newer and a flag here is refused, codex --help shows the current spelling.

Ask for one bounded change

The order below is the lesson. Each step gives you something to compare against in the next.

View full-size diagram

python -m unittest
python tipsplit.py 8650 3

Person 1: $28.84
Person 2: $28.83
Person 3: $28.83

codex "Add an optional tip_percent argument to split_bill in tipsplit.py.
It defaults to 0. The tip is a whole-number percentage of total_cents,
rounded half up to a whole cent, and is added before the bill is split.
Reject a negative tip_percent with ValueError.
Let the command line accept an optional third argument for it.
Keep every existing test passing and add tests for the new behavior.
Do not change anything else."

python -m unittest
python tipsplit.py 8650 3
python tipsplit.py 8650 3 20

git add .
git commit -m "Add an optional tip percentage"

What happened when I ran it

I gave Codex exactly the task above, in the supplied repository, on September 21, 2026. It finished in 61 seconds and reported:

"Added tip_percent=0 with exact half-up rounding, tip inclusion before splitting, and ValueError for negative tips. The CLI now accepts [TIP_PERCENT]. Only tipsplit.py and its tests changed. All 13 tests pass, including every existing test."

It changed two files, tipsplit.py and test_tipsplit.py, and nothing else. Then I ran the three commands myself:

$ python -m unittest
OK

$ python tipsplit.py 8650 3
Person 1: $28.84
Person 2: $28.83
Person 3: $28.83

$ python tipsplit.py 8650 3 20
Person 1: $34.60
Person 2: $34.60
Person 3: $34.60

The second command printed exactly what it printed before the change: the old behavior survived. The third is the new feature, and the arithmetic checks out by hand: a 20 percent tip on $86.50 is $17.30, the total is $103.80, and that divides evenly by three.

The agent's code, its full terminal output, the diff and this output are in agent-run-lesson-01/ in the course materials.

Your run will differ. A model is not a function: it may organize the code differently, write different tests, or make a different mistake. That is exactly why the comparison, not the summary, is what tells you where you stand.

The same steps in an eval repository

This is the task the practice was for. This course has not run it yet: the repository, eval.py and its options below are hypothetical, and nothing here is a measured result.

Suppose your repository has an eval.py that reads a frozen file of model outputs, scores each row, and prints one accuracy line. You want a 95 percent bootstrap interval printed next to it. The task keeps the same parts as the tip task: one function, the exact rule, what must not change, and a boundary.

Add an optional --bootstrap N option to eval.py. It defaults to 0, which prints
nothing new. When N > 0, resample the scored rows N times with random.Random(seed),
using the existing --seed option, and print the 2.5th and 97.5th percentiles
of accuracy after the existing accuracy line.
Do not change how a row is scored, which rows are scored, or the existing output line.
Keep every existing test passing and add tests for the new option.
Do not change anything else.

The baseline is the command you will rerun, with its output saved to a file:

python eval.py results/frozen.jsonl > before.txt

After the change, run the same command again and compare the two files:

python eval.py results/frozen.jsonl > after.txt
diff before.txt after.txt

An empty diff is the evidence that the headline number did not move. The new option's tests tell you the interval code does what Codex thinks it should. Whether a bootstrap interval is the right error bar for your eval is a separate question: Chapter 16 of the book derives the binomial and Wilson intervals and shows when repeated runs of the same case are not new evidence.

What was checked

Executed on September 21, 2026: the repository's baseline and the three commands above, with Python 3.14.3; and the Codex task itself, with codex-cli 0.155.1 signed in through a ChatGPT account, using codex exec --sandbox workspace-write. The install commands, the two sign-in methods, the Auto preset's permissions, /diff, /status and the quoted prices come from OpenAI's Codex documentation, read the same day; the flags were also checked against codex --help. The eval-repository example was written for this page and has not been run; eval.py and its options are hypothetical, not part of the download.