Latest

Sat, Sep 19, 2026Saturday, September 19, 20261 entry

NoteMake it fast and big

Use the Batch API When Your Eval Can Wait a Day

A nightly sweep over 10,000 eval items and the debugging session where you rerun five failures have different deadlines.Quick thoughtopenaibatch-processing

Fri, Sep 18, 2026Friday, September 18, 20261 entry

GuideMake it fast and big

Measure Agent Latency: Time to First Token, Decode and the p95 Tail

Measured on small open models you can run on Colab: prefill that grows with the prompt, decode at a fixed cost per token, a prefix cache 22 times faster, and a queue that explodes near capacity.12 min readlatencyinference-cost

Thu, Sep 17, 2026Thursday, September 17, 20261 entry

NoteMake it fast and big

Put the Rubric First, Then Measure Your Prompt-Cache Hits

Grading 2,000 eval items with the same rubric and the same few-shot examples means sending the same long prefix 2,000 times.Quick thoughtopenaiprompt-caching

Thu, Sep 10, 2026Thursday, September 10, 20262 entries

AnalysisMake it fast and big

Cost per Solved Task: Break-Even Pass Rates, pass@k and Routing

A cheaper model can raise what each solved task costs. Derive cost per solved task as cost per attempt over pass rate, find the break-even pass rate, and route between models.8 min readinference-costagent-evaluation

GuideMake it fast and big

Design a Prompt-Caching Experiment That Measures the Right Thing

Derive when prompt caching pays from Anthropic's write premium and read discount, then run a cold, warm and changed-prefix protocol that keeps cost, context and latency apart.10 min readprompt-cachinginference-cost

Wed, Sep 9, 2026Wednesday, September 9, 20261 entry

NoteMake it fast and big

Give a Reasoning Model the Budget and the Success Criteria

OpenAI's reasoning guide gives short advice. Keep prompts simple and direct. Skip "think step by step," because these models reason internally.Quick thoughtopenaiprompting

That is the first entry in the log.

To browse rather than follow: topics, all articles and all notes. To learn in order: courses and the book.