Latest

Thu, Oct 8, 2026Thursday, October 8, 20261 entry

EssayMeasure models· Food for Thought

A Job ID Is Not a Result: Check the Output Before Marking It Done

A scheduler's job ID, like a deploy hook's 200, says a request was accepted. A five-state model and a small check that marks work done only when the results file exists, from the right checkpoint, with every row.9 min readexperiment pipelinesevaluation infrastructure

Mon, Oct 5, 2026Monday, October 5, 20261 entry

EssayMeasure models· Food for Thought

A Subagent's Report Is Not a Handoff Until the Receiver Can Act

A subagent's report can arrive while everything it learned stays behind. Define handoff acceptance by what the receiver can check and do next, whether it is an orchestrator or a worker resuming a crashed eval run.9 min readhandoffsmulti-agent systems

Thu, Sep 10, 2026Thursday, September 10, 20262 entries

EssayMeasure models· Food for Thought

A Contamination Cutoff Needs a Clock, Not Just a Date

A training cutoff of March 31 is a date without a clock. One post can land on either side of it depending on the time zone. How to pick one clock, size the ambiguous boundary day and keep it out of a held-out split.8 min readevaluationtimezones

EssayMeasure models· Food for Thought

Catch Duplicate Eval Results Before a Dictionary Hides Them

A retry appends a second result for the same case. Build a dictionary keyed by case ID and the pass rate now depends on row order; keep both rows and the error bar shrinks for no reason. Check the list first.8 min readevaluationdata integrity

Thu, Aug 6, 2026Thursday, August 6, 20261 entry

EssayMeasure models· Food for Thought

Clean an Eval Set With a Coding Agent, Then Prove It Dropped Nothing

An eval set merged from three sources has mixed label spellings, near-duplicate prompts and empty references. A coding agent will tidy it. Three counts, computed by your own script, show whether it dropped a row that would fake a gain.8 min readevaluationdata-quality

That is the first entry in the log.

To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.