Test the Date Cutoff in Your Contamination Filter

One common way to reduce contamination in a coding or math eval is a date filter: keep only problems published after the model's training cutoff. That filter is a calendar boundary, and calendar boundaries break in predictable places.

Is the cutoff inclusive? Measured in UTC or in local time? What about a timestamp with a -07:00 offset that falls on the previous day locally but the next day in UTC? Each answer changes which items are in the eval. An item that leaks across the boundary can inflate the score, and no error bar reveals it, because the interval describes sampling noise, not a biased sample.

Write the definition down before any code: "keep items with published_at strictly after 2026-03-31T23:59:59Z, compared in UTC."

The check. Anthropic's best practices for Claude Code recommend asking for a failing test that reproduces an issue before the fix. Ask Claude Code for a test with these fixture items and the expected result for each:

The last row is the one people skip. A bare date is ambiguous, so the filter should either reject it with an error or apply a documented rule, never guess silently.

Run the test against the filter as it stands. If it passes the first time, check that it can fail: flip the comparison to >= and confirm the first row now breaks.

Then log how many items the filter excluded next to the eval score. A reader comparing your number with another group's needs to know the sample differs, and a change in that count between runs is a signal worth a look.

Claude Code documentation.