Tools, Memory, and Multi-Agent Systems, Measured
Reason about agents with numbers: why facts get lost in long contexts, what an eval sweep costs, how reliability decays over a long task, and when a second agent pays. Week 1 derives the arithmetic. Week 2 covers tools, from a command-line wrapper through MCP servers to delegation across a trust boundary. Week 3 builds a ReAct loop and a planner that replans. Week 4 measures memory and retrieval. Week 5 is a case study of a framework debugged in the open. The strongest lessons end in a Colab protocol that usually costs under a dollar. First taught live at Nebius Academy; results from that class are marked as one run.
MethodThis course teaches a discipline. The examples use today's tools; the method is meant to outlast them.
Rod Rivera
Professor
Desk
Every lesson is free. Lessons marked open need no account; the rest open once you register.
What You'll Learn
- Say in one sentence whether a system is an agent, and model it as a policy inside a loop your code owns, with a stopping rule
- Derive why a fact buried among confusable neighbors gets lost while context length alone costs only logarithmically, and measure both on a real model with Wilson intervals and paired comparisons
- Price a model call, an agent session and an eval sweep, derive when prompt caching pays, and choose a model by cost per correct answer instead of by price
- Turn a measured per-step success rate into an honest prediction for a long task, including how its error bar grows as p to the nth power
- Measure a prompt-injection attack against a positive control, and explain why only a boundary outside the model makes a harmful action impossible
- Design tools with five parts and six principles, wrap command-line programs and external APIs safely, serve tools over MCP, and bound what crosses each delegation hop
- Build a ReAct loop by hand and a planner-executor that replans on failure, and count how often each one breaks
- Treat memory and retrieval as measurement problems, defend a memory store at its write path, and tell a run that finished from a run that worked
Prerequisites
- Comfort reading and running Python; every protocol is a short Colab notebook on Python 3.12
- Basic probability and logarithms; the lessons derive what they use, from softmax to binomial intervals
- An Anthropic API key and a few dollars of credit if you want to run the protocols; most cost under a dollar at list price
- Helpful: the book at /book, which builds a working agent from scratch; each lesson links the chapter it builds on
Syllabus
Read the rest of Tools, Memory, and Multi-Agent Systems, Measured
The first lessons need no account. Sign in free to read the rest and keep your place on any device.
