Harness Engineering2026-09-1011 min read
Design a Cache Experiment That Measures the Right Thing
Prompt caching measurement means separating context size, billed usage and latency instead of reading one counter as proof of all three.
Rod Rivera
Topic Hub
Filed under Prompt Caching
1 entry and 2 thoughts filed under this tag.
Harness EngineeringThoughts on Prompt Caching
Measure Cache Hits Instead of Assuming Them
Sep 17 · 2026Two requests can look similar to a person while failing to reuse a cached prefix. Measure the actual usage data. Run a controlled test with a stable prompt and several different synthetic inputs. Record the API's…
openaiprompt-caching
Put Stable Instructions Before Changing Inputs
Sep 16 · 2026If many API requests share the same instructions, keep that stable material together. Changing text at the start of every request can prevent useful prompt-prefix reuse. For a classification workflow, put the fixed task…
openaiprompt-caching
