A byte-perfect quote behind a wrong claim
A research agent writes a literature summary on agent reliability. One line reads: "gpt-4o scores under 25% on τ-bench's retail domain." It cites the τ-bench paper's abstract and quotes this sentence, exactly:
Our experiments show that even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail).
Your citation checker fetches the abstract and finds the quote byte for byte. Green light. The claim is still wrong. The 25% in the sentence is pass^8, the probability that all eight trials of a task succeed. It is not a single-trial score, and the sentence gives no single-trial retail number at all. Nothing in the citation check looked at what the sentence says compared with what the summary claims.
This is the gap between two jobs that get bundled under "citation verification." Provenance asks whether the quoted text really appears in the cited source, unaltered. Entailment asks whether that text, read correctly, supports the specific claim drawn from it. A tool can be exact on the first and blind to the second.
Two questions that sound like one
Provenance is mechanical. Given the source bytes and a claimed quote, you can pin a hash, find the span and compare. No judgment is needed.
Entailment is semantic. The same sentence supports one claim and fails another:
| Claim drawn from the quote | Supported? | Why |
|---|---|---|
| "In τ-bench's experiments, even state-of-the-art function-calling agents succeeded on fewer than half the tasks." | Yes | The sentence says so directly |
| "gpt-4o scores under 25% on τ-bench's retail domain." | No | The 25% is pass^8, a reliability metric over eight trials |
| "Tool-using agents fail most real-world tasks." | No | The sentence reports results on one benchmark, not the world |
| "τ-bench found agents consistent across repeated trials." | Contradicted | The sentence says "quite inconsistent" |
Scroll horizontally to view columns.
All four cite the identical span, and a provenance check returns the identical result for each. Only a reader who understands both the sentence and the claim can tell them apart. The variance article derives pass^k, which is exactly the knowledge the second row needs.
Keep the two findings in the record
The fix is two checks with their results stored separately, never merged into one citation_verified flag. The code below runs on Python 3.12, including Colab. The provenance check is real: it compares bytes against a pinned hash. The entailment labels are supplied by a person, as they would be when you build a labeled set.
import hashlib
from dataclasses import dataclass, replace
# Two sentences of the tau-bench abstract (arXiv:2406.12045), one per line.
SOURCE = (
"We also propose a new metric (pass^k) to evaluate the reliability of agent behavior "
"over multiple trials.\n"
"Our experiments show that even state-of-the-art function calling agents (like gpt-4o) "
"succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail).\n"
).encode()
PINNED = hashlib.sha256(SOURCE).hexdigest()
def exact_match(source_bytes, expected_hash, line_number, quoted_text):
if hashlib.sha256(source_bytes).hexdigest() != expected_hash:
return False
lines = source_bytes.splitlines()
if type(line_number) is not int or not 1 <= line_number <= len(lines):
return False
return lines[line_number - 1] == quoted_text.encode()
@dataclass
class CitationRecord:
claim: str
line: int
quote: str
provenance_ok: bool | None = None
entailment: str | None = None # "supported", "not_supported", "contradicted", or None
def decide(record):
if record.provenance_ok is not True:
return "REJECT: source mismatch"
if record.entailment is None:
return "HUMAN_REVIEW: claim support unknown"
if record.entailment != "supported":
return f"REJECT: quote is real but claim is {record.entailment.replace('_', ' ')}"
return "ACCEPT"
quote = SOURCE.splitlines()[1].decode()
records = [
CitationRecord("agents succeed on under half of tau-bench tasks", 2, quote, entailment="supported"),
CitationRecord("gpt-4o scores under 25% in retail", 2, quote, entailment="not_supported"),
CitationRecord("tau-bench found agents consistent", 2, quote, entailment="contradicted"),
]
for r in records:
r.provenance_ok = exact_match(SOURCE, PINNED, r.line, r.quote)
print(f"{r.claim:50} -> {decide(r)}")
assert not exact_match(SOURCE, "0" * 64, 2, quote) # wrong source version
assert not exact_match(SOURCE, PINNED, 1, quote) # right text, wrong location
assert not exact_match(SOURCE, PINNED, 2, quote.replace("<25%", "<20%")) # altered quote
assert decide(replace(records[0], entailment=None)) == "HUMAN_REVIEW: claim support unknown"
print("provenance controls pass independently of the entailment labels")agents succeed on under half of tau-bench tasks -> ACCEPT
gpt-4o scores under 25% in retail -> REJECT: quote is real but claim is not supported
tau-bench found agents consistent -> REJECT: quote is real but claim is contradicted
provenance controls pass independently of the entailment labelsThe exact match compares bytes in a local copy. It does not authenticate the publisher, prove the source is current, or prove the source is true. A correctly quoted paper can itself be wrong.
APIs that return citations solve provenance, not entailment. Anthropic's Messages API, for example, can return the cited_text and its location in a supplied document. That is valuable provenance, and it still leaves you to check that the cited text supports the sentence built on it.
If your pipeline has a single field for citations, find out whether it measures provenance, entailment or a silent blend. A field that checks provenance alone would accept the gpt-4o claim above. A combined verdict is fine if both underlying findings stay in the record.
Three checkers, three different questions
Once provenance and entailment are separate, you can ask which tool answers which.
- Exact match answers provenance, deterministically. It says nothing about entailment.
- A natural language inference (NLI) model classifies a premise and hypothesis pair as entailment, neutral or contradiction. It is cheap and consistent, but it can miss distinctions that need domain knowledge, such as pass^8 against a single-trial rate.
- An LLM judge reads the quote and the claim with instructions and can use domain knowledge. It is flexible, and it must be calibrated like any judge, as the judge calibration article shows.
Errors compose
A citation passes only if both checks pass. Suppose, hypothetically, that the agent's quotes are exact 98% of the time and your entailment checker wrongly accepts 15% of unsupported claims. An unsupported claim with a real quote then gets through with probability 0.98 × 0.15 = 0.147. Perfect provenance cannot fix a weak entailment check; the pass-through rate is set by the weaker link.
How many labels to measure it
The number to estimate is the entailment checker's false-accept rate on unsupported claims. For a rate near 0.15 and a 95% interval of about ±0.10, the sample size is n = 1.96² × 0.15 × 0.85 / 0.10² = 3.84 × 0.1275 / 0.01 ≈ 49 unsupported pairs. If a checker correctly rejects 42 of 50, its recall on that class is 0.84, with a Wilson interval of about 0.72 to 0.92. Measure the supported class as well, since a checker that rejects everything has perfect recall on unsupported claims and is useless.
A protocol to compare the checkers
This site has not run this comparison. It is a protocol you can run with the analysis fixed in advance.
Collect quote and claim pairs from real agent output on papers you know. Two people label each pair supported, not supported or contradicted, blind to each other; report their kappa and adjudicate. Oversample the subtle cases: metric confusions, overgeneralizations, numbers from the wrong table.
Hold the claim fixed and change only the source: altered bytes, a different version, the same sentence at a different location. Exact match must reject each; the entailment checkers should be unaffected.
Hold the quote fixed and change only the claim, as in the table above. Provenance must pass every time; only entailment should move.
For each checker, recall on each class with Wilson or Clopper-Pearson intervals, the false-accept rate on unsupported claims, and the cost per thousand pairs.
What this does not establish
None of this says any particular citation product is right or wrong. It establishes a design rule and a test: a system that records a real quote with an unsupported claim as a single "verified" result has merged two findings, and the merge is the bug. The falsifying case for this article would be a checker whose provenance verdict reliably predicted its entailment verdict on your labeled set. Build the set and measure it; if the two never disagree on your data, one check may be enough for that data.
Check the decision
Why this matters in a lab
Research agents that read papers and write summaries with citations are a common product and a common research object, and citation faithfulness is one of the first things anyone evaluates in them. An interviewer who asks "how would you evaluate whether an agent's citations are correct?" is checking whether you split provenance from entailment, whether you know what exact match, NLI and model judges each measure, and whether you would label a set and report per-class error rates with intervals. The same split applies to your own writing: every number in a results table should point to a source that states that number, measured that way. Chapter 5 covers the retrieval side, choosing which sources the model sees at all. Chapter 16 and Chapter 17 supply the statistics and the discipline for improving a checker without fooling yourself.
Chapter 5 derives BM25 and the ranking metrics, and measures retrieval against sending everything to a real model.
