Courses

Coding agents, systems and evaluation

The first lessons of each course need no account. Sign in free to read the rest and save your progress.

157 released lessons

12 courses you can start

Free registration

Start a released lesson today, or explore a syllabus for what comes next. Planned lessons do not count as released.

ITAM students: your semester course is here.

12 courses match

Syllabus

Agentic Coding with Cursor for Research Engineers

Review a Cursor agent's multi-file edit to an eval or training script the way a lab reviewer would, and measure how many seeded defects your review catches.

Before you start

Comfort reading Python you did not write
Python 3.12 and uv, set up in lesson 4
Cursor installed, or willing to install it in lesson 2

Available now

12 released lessons

Start learning

Syllabus

Claude Code for AI Research

Run Claude Code on research code and defend every number it produces: reproduced, guarded, measured with an interval, reviewed and recorded.

Before you start

Python you can read, and basic git (commit, branch, diff)
No prior terminal setup required; Lesson 02 starts from a blank machine
A Claude Code plan or API access; this free course does not pay for the tool

Available now

12 released lessons

Start learning

Syllabus

Why Agent Systems Fail, and How a Lab Would Find Out

Find out why an agent system fails, with arithmetic, traces and evidence gates, before a user does.

Before you start

Comfort reading Python and YAML
Basic probability (independent events, a binomial proportion); the course derives the rest
No prior agent-building experience required; the book's agent loop chapter is the companion

Available now

10 released lessons

Start learning

Syllabus

Tools, Memory, and Multi-Agent Systems, Measured

Design, price and measure an agent system's context, tools, reasoning strategy and memory, and report each result with an error bar.

Before you start

Comfort reading and running Python; every protocol is a short Colab notebook on Python 3.12
Basic probability and logarithms; the lessons derive what they use, from softmax to binomial intervals
An Anthropic API key and a few dollars of credit if you want to run the protocols; most cost under a dollar at list price
Helpful: the book at /book, which builds a working agent from scratch; each lesson links the chapter it builds on

Available now

34 released lessons

Start learning

Syllabus

The 2x2 for Agent Design: Workflows, Agents, and Per-Layer Evals

Decompose an agent design into layers, place and price each one, and name the eval loop that shows it works.

Before you start

Comfort calling a model API with tools; this course does not re-teach tool-calling mechanics from scratch
Basic probability (independent events, a binomial proportion); every formula used is derived or linked to its derivation in the book
Enough Python to read and run short standard-library listings; they run on Python 3.12 in Colab
Helpful, not required: the book's chapters on the agent loop (/book/ch03-agent-loop) and evaluation (/book/ch16-evaluation)

Available now

12 released lessons

Start learning

Syllabus

Voice Agents, Not Chatbots: Latency, Barge-In, and Evaluation

Budget, build and evaluate a voice agent's turn-taking, barge-in and task success with measurements someone else can check.

Before you start

Comfort with an LLM calling tools; this course doesn't re-teach tool-calling mechanics
Enough Python to read and run short listings on Python 3.12 (Colab works)
Helpful, not required: chapters 9 (prefill, decode and latency) and 16 (evaluation with error bars) of the book
No prior voice or telephony experience; ASR, TTS and SIP are introduced where they cause a failure

Available now

12 released lessons

Start learning

Syllabus

ZEO ITAM · Autumn 2026

Run a measured dark-factory loop on a business's work, against your own minutes baseline.

Before you start

No programming experience required for Class 1
Access to one real, recurring manual workflow in your own work to use as the semester's case study
Enrolled or auditing the live ITAM Autumn 2026 course helps but is not required to follow the lessons

Lessons released progressively

41 released lessons

Start lesson 1

Syllabus

Inference Economics on NVIDIA NIM: Latency, Throughput and Cost per Token

Derive and measure a model's time to first token, decode rate, throughput and cost per million tokens on NVIDIA NIM, and choose between the hosted API and a self-hosted container with the arithmetic shown.

Before you start

Comfort reading Python and basic algebra; every formula is derived on the page
For the hosted measurements: an NVIDIA API key from build.nvidia.com. NVIDIA's FAQ describes this access as free for prototyping and development, not for production
For Lessons 04 and 05's container steps: an NVIDIA GPU, Docker with the NVIDIA Container Toolkit, and an NGC API key. Without them you can still do the arithmetic and read the protocol
Helpful, not required: the book's chapters on latency (/book/ch09-messaging), queues (/book/ch10-scheduling) and cost (/book/ch19-operation)

Available now

6 released lessons

Start learning

Syllabus

LangGraph for AI Research

Build a LangGraph eval harness and research pipeline that survives a crash, routes a judge's borderline calls to a person, and reports error bars, reliability and cost.

Before you start

Comfort reading Python; every lesson runs real code, and each listing runs on Colab's Python 3.12
No prior LangGraph or LangChain experience; lesson 01 builds the current API from the ground up
Helpful but not required: the probability in chapters 3 and 16 of the profrod.ai book (reliability over n steps, error bars on a pass rate)

Available now

9 released lessons

Start learning

Syllabus

Hosted Inference Economics on Nebius Token Factory: Latency and Cost per Task

Turn a hosted API's dated price list, rate limit and catalog into cost per task and cost per successful task, and measure the endpoint's latency yourself as p50 and p95.

Before you start

Comfort reading Python and basic algebra; every formula is derived on the page
A Nebius Token Factory API key (sign-up at tokenfactory.nebius.com) to run the measurement protocol; the arithmetic needs no key
Helpful, not required: the companion course on NVIDIA NIM, which covers the self-hosted side, and the book's chapters on latency (/book/ch09-messaging) and cost (/book/ch19-operation)

Available now

5 released lessons

Start learning

Syllabus

OpenAI Codex for Research Engineers: First Task to Reviewed Change

Before a coding agent touches eval code, record a baseline, give it one bounded change, read the diff, and confirm that the outputs you did not ask it to change are unchanged.

Before you start

Basic Python, enough to read a short function and a unit test
Helpful, not required: some experience running evaluations or experiments in Python
A terminal, Git, and Python 3.9 or newer
A ChatGPT plan that includes Codex, or an OpenAI API key billed at OpenAI's standard API rates

Available now

2 released lessons

Start learning

Syllabus

Evaluating Coding Agents: Acceptance the Agent Cannot Redefine

Grade a coding agent's work with acceptance it cannot redefine, and report a pass rate over a fixed task set with a 95 percent interval.

Before you start

Python, enough to read a 100-line script and a unit test
A terminal and Python 3.9 or newer. No container engine, no API key and no account are needed for the opening unit
Enough probability to follow a mean and a variance; the lesson derives the rest
Helpful, not required. The Codex course in this track, or the Claude Code course, for operating a coding agent

Available now

2 released lessons

Start learning

New here?

Start with the book. One model call, derived and run on Colab.

Chapter 1 derives tokens, probabilities and sampling, then makes one request to a model in a Python 3.12 notebook. The later chapters build on it.

Free to read, no account needed. Prefer a hands-on exercise first? Reconcile two exports against a local checker.

Start with Chapter 1

Am I ready for a lab?

Twelve questions on the work frontier labs hire for. See where you stand and which skill to train next.