The baseline that never expired but was already wrong
A research team's agent writes the nightly eval summary. It keeps a small memory of facts it needs every night, including which checkpoint is the team's current baseline for the math eval. The record says ckpt-0412. It has a timestamp and no expiry, because nobody thought a baseline needed one.
On Thursday the eval team re-runs the baseline with a fixed grader and registers ckpt-0519 as the new baseline. An agent that read the registry writes a fresh record. Nobody retires the old one.
Friday night, a different agent instance retrieves the old record, because it matches "current math-eval baseline" slightly better by whatever similarity the store uses. Nothing about it looks broken. The summary compares last night's run against the old baseline and reports a gain that mostly comes from the grader fix. Someone quotes it in a meeting.
That is a fabricated result, and no model hallucinated it. The memory store served a fact that had been true and was not any more. Figures in this example are fixture values, not a real eval.
Two ways a fact goes stale
A timestamp answers "when was this written." It cannot answer "is this still true," because a fact can stop being true for two unrelated reasons:
- Time passes: A cluster quota valid through the end of the quarter expires on a date you can know in advance. A
valid_untilfield catches it. - The source is revised: The baseline changed because the registry changed, on a day nobody could have predicted. Only a comparison with the source's current revision catches it.
A store that checks only dates will serve the second kind of stale fact with full confidence. A record with a far-future valid_until can be wrong today.
Four kinds of memory, four retirement rules
Not every persistent note is the same kind of thing. Chapter 5 makes the same point for a single customer's preferences; a research team's agent has at least four kinds.
| Memory type | Example | Who can change it | What retires it |
|---|---|---|---|
| Operational fact | Current baseline checkpoint, cluster quota | The system of record (eval registry, scheduler) | A newer source revision, or its expiry date |
| Procedural instruction | "Report every eval delta with a 95% interval" | A named person, through a versioned document | A new version of the instruction |
| Investigation note | "lr 3e-4 diverged at step 2k on the 1B config" | The agent or person who wrote it | Nothing; kept as history under the retention policy |
| Job state | "Sweep 14 is blocked on quota" | The current job owner | The job's own state machine |
Scroll horizontally to view columns.
The investigation note and the operational fact have opposite lifespans. You want the divergence note for months, because it explains a decision. You want a superseded baseline to stop being selectable as current immediately, while still answering "what did we believe last week, and why."
The record and the rule
Give every operational fact five fields beyond its value: owner, source_revision, valid_until, supersedes and status. Then choose the current fact by rule, not by similarity. The rule reads the owner's current revision from the owner's system, never from whichever memory record won retrieval.
from dataclasses import dataclass, replace
from datetime import date
from typing import Optional
@dataclass(frozen=True)
class Fact:
fact_id: str
fact_key: str
value: str
owner: Optional[str]
source_revision: str
valid_until: Optional[date]
supersedes: Optional[str]
status: str # current, superseded, expired, historical_only
records = [
Fact("baseline-v17", "baseline.math-eval", "ckpt-0412", "eval-registry",
"registry@r17", None, None, "superseded"),
Fact("baseline-v18", "baseline.math-eval", "ckpt-0519", "eval-registry",
"registry@r18", date(2026, 12, 31), "baseline-v17", "current"),
]
def select_current(fact_key, records, today, authority):
# authority is read from the owner's system of record, never from a memory record
if not authority or not authority.get("owner") or not authority.get("source_revision"):
return {"code": "current_authority_unavailable"}
candidates = [r for r in records if r.fact_key == fact_key]
if any(r.status == "current" and not r.owner for r in candidates):
return {"code": "owner_missing"}
eligible = [
r for r in candidates
if r.status == "current"
and r.owner == authority["owner"]
and r.source_revision == authority["source_revision"]
and (r.valid_until is None or r.valid_until >= today)
]
if len(eligible) == 1:
return eligible[0]
if not eligible:
return {"code": "no_current_record", "candidates": [c.fact_id for c in candidates]}
return {"code": "authority_ambiguous", "candidates": [c.fact_id for c in eligible]}
authority = {"owner": "eval-registry", "source_revision": "registry@r18"}
today = date(2026, 9, 30)
print(select_current("baseline.math-eval", records, today, authority).value)
stale_but_marked_current = replace(records[0], status="current")
checks = {
"superseded v17 left current": select_current("baseline.math-eval", [stale_but_marked_current], today, authority),
"owner missing": select_current("baseline.math-eval", [replace(records[1], owner=None)], today, authority),
"two current at r18": select_current("baseline.math-eval", records + [replace(records[1], fact_id="baseline-v18b", value="ckpt-0520")], today, authority),
"last valid day": select_current("baseline.math-eval", records, date(2026, 12, 31), authority),
"day after": select_current("baseline.math-eval", records, date(2027, 1, 1), authority),
"registry unreachable": select_current("baseline.math-eval", records, today, None),
}
for name, result in checks.items():
print(f"{name:28}", result.value if isinstance(result, Fact) else result["code"])ckpt-0519
superseded v17 left current no_current_record
owner missing owner_missing
two current at r18 authority_ambiguous
last valid day ckpt-0519
day after no_current_record
registry unreachable current_authority_unavailableI ran it with Python 3.12 on September 30, 2026. Read the second line carefully: even when the old record is wrongly left marked current, its revision r17 does not match the registry's r18, so it cannot be selected. The two expiry triggers are checked independently, and the date boundary is inclusive through valid_until in one declared timezone. When the registry cannot be reached, the function says so instead of falling back to whatever it has.


The idea is not new to memory systems. LangChain's post on self-correcting memory describes a knowledge base that drifts as the underlying code changes, and ties each claim to the source version it came from rather than to a re-read schedule. Same mechanism: staleness follows the source, not the clock.
Who may write, supersede and retire
The owner field only works if "who may change this" has an answer.
The eval registry owns baselines, the scheduler owns quotas, a named person owns each procedural instruction. The owner is a role or a system, not a running agent instance, so it survives restarts.
A registry revision, a document version, a snapshot ID. A fact with no traceable source can never be checked against one.
Writing the new record and marking the old one superseded happen atomically, or through a conditional update of a single current pointer. Two unprotected writes let a reader see both or neither.
If selection finds two eligible current records, the output is an escalation naming the owner, not a similarity score or a recency guess.
The escalation is a small, concrete message. Until the owner resolves it, the agent may read history, so it can still explain what it knew, but it may not write a derived fact from either candidate:
escalation = {
"code": "authority_ambiguous",
"fact_key": "baseline.math-eval",
"candidate_refs": ["baseline-v18", "baseline-v18b"],
"owner": "eval-registry",
"next_action": "confirm the registered baseline",
"allowed_until_resolved": ["read_history"],
"write_allowed": False,
}That freeze is what keeps an ambiguous baseline out of tonight's summary. If no owner is configured for a domain, route to a fallback owner rather than letting the newest-looking record win, because "newest-looking" is exactly how the old baseline got through.
A record with valid_until next year can still be wrong today if its source was revised. Check the source revision independently of the date, every time the fact is used for a current answer.
Check the decision
Why this matters in a lab
Research agents are starting to keep memory across runs: baselines, known-bad configurations, which dataset version a result used. A stale fact in that memory does not look like a model error. It looks like a result, and results travel. The failure in the opening story, a comparison against the wrong baseline, is the kind of mistake that ends up in a table.
In an interview or a design review, the strong answer to "how does your agent's memory stay correct" names the owner of each fact type, the two expiry triggers, the selection rule, the transaction that supersedes, and what happens on conflict. A record with an owner and a revision is also what makes a result reproducible months later: you can say which baseline the agent believed and why.
For the memory design that surrounds this, the Tools, Memory, and Multi-Agent Systems course covers memory types and shared memory across agents, and what an agent remembers after a restart shows the storage layer this sits on.
What would prove this design wrong
The design fails its own test if selection ever returns a superseded record, silently picks between two current records, treats a missing owner as a valid one, or returns a value when the owner's system is unreachable. Each of those is a line in the fixture above; run them against your store.
The fixture is an in-memory list. It says nothing about concurrent writers, replication lag or durable commits; those need tests against the datastore you actually use. To measure how often stale facts reach answers in a real store, retiring stale memory when the source changes gives a protocol with recall@k.
Derive BM25 and the ranking metrics, measure retrieval on a real model, then build preferences you can correct in one transaction.

