Key a Cached Answer to Its Reader's Current Access, Not Just the Query

Nothing changed except who may know

A research assistant answers "summarize last night's eval sweep" for Ana. The answer draws on an eval report, d3, that only the eval team may read. The system caches the answer so the next person to ask gets it without another retrieval and model call.

Then Ana moves teams, and her access to d3 is revoked. She asks the same question, word for word. The question text has not changed. The document has not changed. If the cache key was hash(query), the system finds a hit and hands back the old answer. Retrieval was correct. Authorization was not, and no amount of re-embedding would notice.

Two caches that look alike

Prompt caching is a provider feature that reuses the computation behind a repeated prefix of input tokens. Anthropic's prompt caching documentation describes a default 5-minute lifetime and states that caches are isolated between organizations and, on the Claude API, per workspace. That isolation separates your workspace from others. It knows nothing about the people inside your application.

Answer caching is something you build. You store a finished response so a repeated question skips retrieval, reranking and generation. The moment the key is only the question, you have promoted a cost optimization into an authorization decision. Your application has to enforce who may read the cached answer, on the hit path as well as the miss path.

A key that carries authority

Widen the key so it identifies the answer's authority as well as its content, and keep a current check before release. One design, offered to build and test rather than as a measured result:

import hashlib

auth_revision = {"ana": 7, "bo": 3}  # bumped by the system that grants and revokes
cache = {}
generations = 0


def cache_key(principal, query, doc_revisions, generator="summarizer-v4"):
    deps = ",".join(sorted(doc_revisions))
    raw = f"{principal}|{auth_revision[principal]}|{query}|{deps}|{generator}"
    return hashlib.sha256(raw.encode()).hexdigest()[:12]


def answer(principal, query, doc_revisions):
    global generations
    key = cache_key(principal, query, doc_revisions)
    if key in cache:
        return "hit", key
    generations += 1  # stands in for retrieval, reranking and a model call
    cache[key] = f"answer drawn from {doc_revisions}"
    return "miss", key


print(answer("ana", "eval summary", ["d3:v2"]))
print(answer("ana", "eval summary", ["d3:v2"]))
auth_revision["ana"] += 1  # Ana loses access to d3
print(answer("ana", "eval summary", ["d3:v2"]))
print("generations:", generations)

('miss', 'ff83b37bea79')
('hit', 'ff83b37bea79')
('miss', '1eed9d01504a')
generations: 2

I ran it with Python 3.12 on September 30, 2026. The revocation bumped Ana's revision from 7 to 8, so the old entry no longer matches and nobody had to find and delete it. A miss is not enough on its own: the regenerated answer must come from retrieval that applies current permissions, which will no longer return d3. Filtering retrieval by permission before the prompt covers that half. The fixture passes the same dependency list only to keep the example short.

View full-size diagram

What the key costs, in tokens

Safety here is paid for in generations, and you can count them. Say 200 researchers each ask for the same sweep summary, and one answer costs 8,000 input tokens of retrieved context plus 500 output tokens: 8,500 tokens per generation.

• Key on the query only: 1 generation, 8,500 tokens. Unsafe.
• Key per principal: 200 generations, 200 × 8,500 = 1,700,000 tokens.
• Key per permission class: if those 200 people fall into 3 distinct sets of readable documents for this corpus, 3 generations, 3 × 8,500 = 25,500 tokens.

Multiply by your model's per-token prices to turn tokens into dollars. The middle line is 200 times the first, and the third recovers most of the saving. It is safe when two readers in a class can read exactly the same documents in scope, because then retrieval returns the same passages for both. Weaker sharing rules, such as "B may read everything this answer cited," are also safe for access but can cost recall: B might have been entitled to a better document that A could not see.

The part a static test misses: revocation mid-flight

Ask, revoke, ask again tests only a revocation that completes between requests. Real systems have requests in flight. Picture Ana's request past retrieval, holding a generated answer, about to be cached and returned, when an administrator revokes her access and her revision moves from 8 to 9.

def admit(pending, current_revision):
    # Run inside the same transaction or lock as the revocation write.
    return "release" if pending["auth_revision"] == current_revision else "authorization_changed"


pending = {"principal": "ana", "auth_revision": 8, "answer": "fixture answer"}
print(admit(pending, auth_revision["ana"]))
auth_revision["ana"] += 1  # a revocation commits while the answer is pending
print(admit(pending, auth_revision["ana"]))

release
authorization_changed

The comparison is one line. The discipline is where it runs: immediately before the answer is released or written to the cache, not when the request started. A check only at the start passes this case, because Ana still had access then. And the admission step and the revocation write need a shared transaction, lock or equivalent, or a revocation can commit between the check and the release. Bytes already sent to the reader cannot be recalled; streaming responses need their own policy.

View full-size diagram

Keep a positive control: if the revision does not change, the pending answer must be released. A suite that tests only the revoked path can pass a system that refuses everyone. And check scope: invalidation for Ana's revocation should touch Ana's entries, not every entry that mentions d3, unless the change is document-wide, such as d3 being deleted.

Check the decision

Why this matters in a lab

Inference cost is a design constraint on any deployed agent, and caching is the first lever people reach for. Applied-AI work asks you to pull it without breaking access control. A strong interview answer separates provider prompt caching from application answer caching, puts authority in the key, rechecks at release, and then does the arithmetic above: what per-reader keys cost in generations, and how permission classes win most of it back. That combination of a correctness argument and a token count is what a reviewer looks for in a work sample.

What would prove this design wrong

If a fresh request after revocation, with no timing involved, still returns the old answer, the key has failed; that is the cheap deterministic test. If the paused-request test releases the answer, the recheck is in the wrong place or not coordinated with the revocation. If the permission-class key ever serves an answer built from a document one member of the class cannot read, your class definition is wrong.

Nothing here was run against a production cache; the fixtures show the logic, and the token numbers are arithmetic on stated assumptions.