Time Each Stage of an Agent Run Before You Optimize It
An eval sweep that takes all night limits how many ablations you can run in a week, and "why is this agent slow?" is a fair interview question for an applied-AI role. A good answer starts with a trace, not a guess about the model.
The protocol. Wrap each stage of one agent run in a timer: input preparation, retrieval, each model request, each tool call, validation and every retry. For model requests, log time to first token and total time separately. Run 50 representative cases drawn from the eval set, including the ones that trigger retries.
What to report. For each stage, give the median and the slowest case, and count how often that stage is the largest share of the run. Keep the acceptance check fixed while you compare configurations, or a faster run may simply be a sloppier one.
The arithmetic. A model request is roughly prefill plus decode: time to first token, then about one step per output token, so decode time grows with output length. OpenAI's guide says that cutting 50% of output tokens may cut about 50% of latency, while cutting 50% of the prompt may only improve it by 1 to 5%. Stages in series add up. Two independent stages run in parallel cost the maximum of the two, not the sum. The book's messaging chapter derives the prefill and decode split.
Put an interval on your counts. If retrieval is the largest stage in 30 of 50 runs, p = 0.60 and the 95% interval is 1.96 × √(0.60 × 0.40 / 50) ≈ 0.14, so 0.46 to 0.74. That is enough to act on. A 27-of-50 split between two stages is not.
This site has not run this protocol on a specific pipeline, so no stage breakdown is claimed here. The expected result is a table you can defend: the stage that owns the wall clock, with its count and interval, and the change you will try first.
