Design a Prompt-Caching Experiment That Measures the Right Thing

The question an applied-AI team will ask you

An agent sends the same 20,000-token system prompt and tool list on every call. Someone proposes prompt caching. Will it pay, by how much, and how would you show it?

That is a fair interview question for an applied-AI role, and a fair first week task. A good answer has three parts: the arithmetic from the provider's price schedule, a clear statement of what caching does and does not change, and an experiment design that would catch you if the arithmetic were wrong. This article builds all three for the Claude API. The experiment is written as a protocol. It has not been run for this article yet, so no result below is a measurement unless it says where the measurement came from.

What a cache hit changes, and what it does not

Anthropic's prompt caching documentation describes reuse of an unchanged prompt prefix. The API matches the start of the request against prefixes it processed recently, in the order tools, then system, then messages. A change at one level invalidates that level and everything after it.

Each response reports three input counts, and the documentation gives their sum:

input_tokens counts only the tokens after the last cache breakpoint. The three categories are disjoint, so add them; do not subtract reads from a total the way you might for another provider's API.

Caching changes what you pay for the prefix and how much prefill work the server repeats. It does not shrink the prompt. Cached tokens still count toward the context window. Three questions therefore need three instruments:

View full-size diagram

Derive when caching pays

The price schedule sets the economics. On Anthropic's pricing page, read on 2026-09-30, a cache write costs 1.25 times the base input price for the five-minute lifetime and 2 times for the one-hour lifetime. A cache read costs 0.1 times the base input price on most models. The exceptions on that date were Claude Opus 5.5 at 0.05 times and Claude Fable 5.1 and Mythos 5.1 at 0.025 times.

Call the base input price p, the write multiplier w and the read multiplier r. A prefix of P tokens is sent in N requests, and every request after the first arrives while the entry is alive. Without caching, the prefix costs N·P·p. With caching, it costs one write and N − 1 reads:

For the five-minute write, (1.25 − 0.1)/(1 − 0.1) = 1.28, so two requests are enough: one read pays for the premium. For the one-hour write, (2 − 0.1)/0.9 = 2.11, so you need three requests, two reads. That matches the pricing page's own statement. As N grows, the ratio of cached to uncached cost falls toward r, so 0.1 is the floor on most models.

Now put numbers on the opening question. Claude Sonnet 5.5 was listed at $2 per million input tokens, $2.50 for five-minute writes, $0.20 for reads and $10 for output. Fifty requests share a 20,000-token prefix within one cache lifetime:

The cached prefix costs 12.3% of the uncached one. The assumption doing the work is that all 49 reads land while the entry is alive. The five-minute lifetime is refreshed by each hit, so a steady agent loop keeps it warm, while a user who pauses for ten minutes does not.

Cost from the usage fields

Once a response returns, cost is a dot product of disjoint counts and dated rates. The function below makes the mapping explicit and refuses a count with no rate, so a new category cannot silently drop out of the bill.

def estimated_cost(usage, rates_per_million):
    """usage and rates share keys; counts must be disjoint categories."""
    missing = set(usage) - set(rates_per_million)
    if missing:
        raise ValueError(f"Missing rates for: {sorted(missing)}")
    if any(count < 0 for count in usage.values()):
        raise ValueError("Token counts cannot be negative")
    return sum(count * rates_per_million[k] for k, count in usage.items()) / 1_000_000

# Claude Sonnet 5.5 list prices in USD per million tokens, read 2026-09-30
rates = {"input": 2.00, "cache_write_5m": 2.50, "cache_read": 0.20, "output": 10.00}

uncached = {"input": 5_200, "output": 50}
cold = {"input": 200, "cache_write_5m": 5_000, "output": 50}
warm = {"input": 200, "cache_read": 5_000, "output": 50}
for name, usage in [("uncached", uncached), ("cold", cold), ("warm", warm)]:
    print(name, round(estimated_cost(usage, rates), 6))

It prints uncached 0.0109, cold 0.0134 and warm 0.0019. Check the warm line by hand: 200 × 2 + 5,000 × 0.2 + 50 × 10 = 400 + 1,000 + 500 = 1,900 millionths of a dollar. The cold call costs more than not caching; the warm call costs about a sixth. That ordering is the prediction the experiment will test.

The protocol: cold, warm and changed prefix

View full-size diagram

How to read the results

Cost comes straight from the usage fields and the dated rates, so it needs no statistics: each condition has a deterministic price given its counts. Latency does. Report the median and p95 of time to first token per condition, with the trial count. With 30 trials, the p95 rests on the slowest one or two values, so treat it as a rough indication; the latency article derives how many trials a percentile needs.

Expect the cost saving to be large and reliable, and the latency saving to be smaller and noisier. Chapter 21's Claude run is a useful reference: stable-first turns on Claude Sonnet 5.5 took a median of 1,075 ms to the answer against 1,377 ms volatile-first, while Haiku 4.5 showed about 0.8 seconds either way, because time over an API also includes the network and the provider's queue. A saving you can count on is the bill.

Checkpoint: the break-even point

Check the decision

What an interviewer would ask

Expect follow-ups that test whether you can separate the three quantities under pressure. Does caching let you fit more into the context window? No: cached tokens count toward it. Why did the warm call show zero reads? Check the minimum length, the lifetime, and whether any byte before the breakpoint changed, including tool order. Would you use the one-hour lifetime for a batch that finishes in two minutes? No: you pay 2x for writes and never use the extra time. How would you know the latency gain is real? Interleaved repeated trials, medians and percentiles with their counts, and the output length held fixed.

In a lab the same reasoning prices every eval sweep and agent run that shares a long prefix. A candidate who can derive the break-even point, predict the usage fields and design the control that could prove them wrong is doing the work the team does.

What would prove this wrong

The protocol is built to be falsified. If the warm condition's billed cost does not come in near a sixth of the uncached one at these counts, check the rates you applied and the read count before trusting either. If reads appear in a condition where you predicted a full invalidation, your model of the prefix hierarchy is wrong for that request, and the serialized request will show why.

A dated check before you trust any of this later

Documentation pages change without notice. If you read this well after September 2026, reopen the sources before reusing a number.