Run the application before automating changes

Before you ask an agent to change an application, make sure you can tell whether that application works. Our application updates a synthetic customer’s email address. The interesting case is a request that succeeds at the receiving system but loses its response on the way back.

By the end of this lesson, you should be able to explain why “I didn’t receive a receipt” and “the update didn’t happen” are different statements.

Prepare your machine

Download the complete project and unzip it. Open a terminal in the dark-software-factory-v1 folder. You need uv on macOS or Linux; on Windows, work inside WSL. On Linux you also need bubblewrap (bwrap), which most distributions package. macOS has its sandbox built in. No Python packages are installed: the exercise uses only Python’s standard library.

uv --version
uv run python --version

The second command reads the project’s pyproject.toml and uv.lock and prints a Python 3.13 version. If you have no Python 3.13, uv downloads one into its own folder, so this first step may need an internet connection. The application checks need none.

Run the baseline

uv run factory.py check reference/b0.py --contract b0 --out runs/baseline-01

Expected result: the command exits with status 0 and prints "passed": true. Open runs/baseline-01/report.json. Every check records its name, actual result, expected result and pass state. Keep this file. A fresh output directory is required for each attempt so a rerun cannot erase your earlier evidence.

The controller starts three roles: the application, a simulated customer-record system and a test client. The application runs inside your operating system’s sandbox: Seatbelt on macOS, bubblewrap on Linux. It can write its own state folder and nothing else. It cannot read your home folder, and it has no network; it reaches the simulator through a socket file. It receives no model credentials. If your machine has neither sandbox, the controller refuses to run the application at all. The test controller retains access to the simulator’s private ledger. The application does not receive that ledger’s administrator token.

The baseline has deliberately modest behavior. It validates the request, checks that fixture account A cannot update B, forwards a valid update and records a confirmed receipt. It does not yet prevent duplicate updates when the same request is retried.

Read the two sides of the write

Find ordinary_write:receipt in the report. The checker compares the application’s receipt with the receiving system’s ledger. A success response alone would let an implementation pass by inventing a receipt.

Now find lost_reply_unknown. The simulator committed the update and then closed the connection without a response. The correct application result is HTTP 202 with state: unknown. The ledger still contains the write.

Observation	What it establishes
The receiving system rejects the request before writing	No effect occurred in this defined failure mode.
The receiving system records an effect and returns its receipt	The checker can match the response to that effect.
The connection drops after the write	The caller lacks confirmation, although the effect exists.

These are controlled teaching conditions. A real API’s documented failure semantics may differ. You must establish those semantics before treating an HTTP error as proof that no write happened.

Check your understanding

Question: The application returns unknown. Should the user press Retry immediately?

Answer: The baseline has no duplicate protection. A retry can perform the update again. First establish whether the original operation took effect, or add an operation identity that makes retries safe. We will add that identity next.

Your evidence: Save the baseline report and write two sentences explaining which system knows that the update happened. If the run failed, keep its report and resolve the setup or application failure before continuing. A missing check is not a pass.

Continue to specifying the change.