Parse Codex's Final Result, Not Its Progress Stream
A sweep driver calls codex exec once per config and builds a results table. If it parses everything the command prints, two things go wrong. A progress line lands in a results cell, or a failed config leaves no row and nobody notices.
The second is worse. Failures are rarely spread evenly across a sweep: the high learning rates diverge, the long-context configs time out. Drop those rows silently and the table describes a different sweep from the one you ran, with the hard cases removed. That is selection bias, and no single-run check catches it, because each surviving row is fine.
The first problem has a documented fix. Per OpenAI's non-interactive guide, codex exec sends progress to stderr and only the final agent message to stdout, and with --output-schema and -o the final response is written to a file in the shape your schema defines. Build the table from the -o files and nothing else, and keep stderr in a log next to each one:
codex exec --output-schema ./row.schema.json -o "rows/$cfg.json" \
"Summarize configs/$cfg.yaml as one results row." 2> "logs/$cfg.log"
jq -e '.status and .metric != null' "rows/$cfg.json"jq -e exits nonzero when the expression is false or null.
The check: account for every config. For each config the driver writes either a row or a rejection line naming the config and the reason. At the end it asserts:
rows written + rows rejected = configs launchedTest that with three cases: a normal run, a row.json with a required field deleted by hand, and a run you make fail. The first should produce a row, the other two rejections, and the count should hold. Report the rejection count with the table. A reader who sees "38 of 40 configs, 2 diverged" can judge the result; one who sees 38 rows cannot.
This note is about the sweep as a whole. Whether one run's output is safe to use (its exit status, its schema, and numbers that agree with its logs) is the per-run gate in gating a Codex step on its status, schema and numbers.
