RAG Access Control: Stop Leaks Before the Prompt Builds

The document that shouldn't have been there

Consider a hypothetical shared document assistant. Two people, call them Ana and Bo, share a company FAQ. Each of them also has one private document: Ana has a salary letter, Bo has a performance review draft. Bo asks the assistant, "What's in my recent review?" The retrieval pipeline embeds the query, pulls back the top twenty candidate passages by vector similarity, reranks them, and hands the top few to the model. The model writes an answer. Suppose, for a moment, that answer never mentions Ana's salary letter. Did the system behave correctly?

Not necessarily. The question that matters is not what the model chose to write about. It's what documents made it into the context window in the first place. If Ana's salary letter was among the passages retrieved and reranked, then handed to the model alongside Bo's own material, the boundary was already crossed the instant that passage entered the prompt. A well-behaved language model choosing not to repeat it back is not a permission check. The application has already failed its context-access rule.

This is the core distinction this article works through: relevance and authorization are two separate questions, answered by two separate mechanisms, and a retrieval-augmented generation (RAG) system that only checks the first one has no access control at all. RAG access control means constraining the candidate set to what the caller is allowed to see, before ranking, before reranking, before the prompt is assembled. Get that ordering wrong and everything downstream, including a clean-looking answer, is not evidence of safety.

Why relevance and authorization are different filters

A retrieval pipeline typically has three stages: embed the query into a vector, search an index for the nearest candidate passages, then rerank those candidates by a model that scores query-passage pairs more precisely than raw vector distance can. Each of those stages answers the question "how relevant is this passage to the query?" None of them, by default, answers "is this caller allowed to see this passage?"

The NVIDIA NeMo Retriever documentation describes reranking as exactly this: a distinct pipeline stage that rescores candidate passages for relevance, separate from the embedding step that produced them (NVIDIA: NeMo Retriever text reranking overview). Nothing in that description implies document-level permission checking; reranking optimizes for "which of these is the best match," not "which of these is this user allowed to have." If your access control lives only in the prompt instructions ("don't mention documents that aren't the user's"), you've asked a relevance-scoring stage to also be a security boundary, and it was never built for that job.

The MCP authorization specification describes validation of access tokens for a protected resource. That does not define which internal documents each caller may read. Establish request identity through the authentication boundary, then enforce document permissions. A shared service credential, in particular, must not be mistaken for Bo's individual identity. This article assumes trusted authenticated caller context is available to the retrieval function.

Where the filter has to sit

Apply current permissions at the retrieval boundary and recheck results before constructing downstream requests. Where the vector database supports a correctly enforced authorization filter, constrain the query there. A trusted search service may internally inspect index identifiers while locating permitted results; the important output boundary is that unauthorized passage text does not reach the caller's reranker or generation context.

The application must know which component enforces the filter. Filtering an answer after generation is too late for a rule that forbids unauthorized text from entering model context. Filtering after a hosted reranker has received the passages is also too late for that reranker boundary. Keep any broader internal index work inside a service authorized to process those records.

Building the test, not just the filter

Suppose you build a small in-memory fixture to check this reasoning, the kind of toy example useful for teaching the mechanism, distinct from anything running against production storage or a real vector database with concurrent writers. Three documents: an FAQ entry both users may read, Ana's salary letter, and Bo's review draft. An allow-list dictionary maps each user to the document IDs they may see.

from dataclasses import dataclass

@dataclass
class Document:
    doc_id: str
    text: str

# Synthetic document text, never real personnel records.
documents = [
    Document("faq-1", "Reset your password from the settings page."),
    Document("salary-ana", "Synthetic Ana-only compensation marker."),
    Document("review-bo", "Synthetic Bo-only review marker."),
]
current_permissions = {
    "ana": {"faq-1", "salary-ana"},
    "bo": {"faq-1", "review-bo"},
}

def allowed_ids(user):
    if user is None:
        raise PermissionError("Authenticated caller required")
    return set(current_permissions.get(user, set()))

def retrieve_candidates(query, authorized_docs):
    # Mock search: all supplied documents are relevant to this fixture.
    return list(authorized_docs)

def filter_by_authorization(candidates, user):
    allowed = allowed_ids(user)
    return [doc for doc in candidates if doc.doc_id in allowed]

def build_context(query, user):
    authorized = filter_by_authorization(documents, user)
    candidates = retrieve_candidates(query, authorized)
    reranker_input = filter_by_authorization(candidates, user)
    reranked = list(reversed(reranker_input))  # mock relevance ordering
    return filter_by_authorization(reranked, user)

bo_context = build_context("What's in my recent review?", "bo")
assert {doc.doc_id for doc in bo_context} == {"faq-1", "review-bo"}
assert "salary-ana" not in {doc.doc_id for doc in bo_context}

The expected Bo context contains the shared FAQ and Bo's review, in the mock's reversed relevance order. Ana's document is excluded before the mock search and reranker. The permission map is separate from indexed text so a permission change can take effect without rewriting the documents.

This is an in-memory fixture with no embedding model, vector database or external API. It exercises the application boundaries using scripted search and ranking. A production test must capture what each real component actually receives; matching this helper's output does not establish that an adapter passes its filters correctly.

The trap hiding in caching

A cache can bypass a correct filter if its caller assumes cached results are already authorized. For example, an authorized result cached for Ana under a query-only key must not be returned directly for Bo's identical query. A cache hit can safely supply candidates only if the current caller's permission check still runs before those passages leave the trusted retrieval service.

Partitioning by caller or tenant reduces cross-user reuse, but identity alone does not handle permission changes for the same person. Include an authorization revision in the cache identity where available, invalidate affected entries, and recheck current permissions at the context boundary. Also preserve document identity and version so a cached ID cannot silently refer to different text.

The two branches below converge on the same current-permission check. A cached result created for Ana may omit Bo's own relevant document even after Ana's private material is removed. That is a retrieval-quality problem: reuse of a restricted candidate set can lose recall. Partition the cache appropriately or perform fresh retrieval when the candidate set is unsuitable; permission filtering alone does not restore missing candidates.

Access can change after indexing. If a manager loses access to Bo's review, stale index metadata must not continue authorizing it. Use an authoritative permission source or a documented, enforced authorization-version mechanism. Define how quickly revocation takes effect and what happens when that source is unavailable; this example denies use rather than treating a failed permission lookup as unrestricted access.

The NVIDIA NIM course's retrieval lesson explains the embedding and reranking stages this check surrounds. Its relevance examples are useful for understanding the pipeline, but they do not establish that a particular vector database or reranker enforces your document permissions.

Running the acceptance check end to end

In this deterministic fixture, compare against the exact expected set. In real retrieval, relevant results may omit some permitted documents, so require the returned set to be a subset of the current allow-list and test relevance separately. Neither check can be replaced by reading the final prose for signs of leakage. An answer that happens not to mention the salary letter tells you the model chose not to repeat it. It does not tell you the letter was absent from what the model read.

Check the decision

What would show this filter is actually working

A claim this specific needs a way to be wrong. The filter fails its job if any of the following happen in a real test: the authorized retrieval output contains a document outside current permissions; the reranked ID set contains one after filtering was supposed to run; a cache hit returns a candidate list from a different user's session without the filter re-applied; or a document whose access was revoked after indexing still appears because the filter trusted stale index metadata instead of a live permission lookup. Each of those is a concrete, checkable condition, not a vague sense that something might be wrong. Building the fixture in this article and deliberately breaking the filter, by disabling step three above, is a useful way to see false negatives your test would need to catch before trusting the same shape in a live system.

None of this claims that any specific vector database, cache layer, or hosted reranking API enforces access control for you by default. The NeMo Retriever documentation describes reranking as a relevance-scoring stage, not an access-control layer, and nothing in the Model Context Protocol's authorization specification implies that a valid credential grants blanket access to every resource an application might expose. The filter has to be your application's own responsibility, applied at the boundary between "documents that exist" and "documents this caller may read," before anything gets reranked or handed to a model.