Course in preparation · Coming soon
PyTorch for Research Engineers
Write the training loop a lab team would trust. Build tensors, autograd and a small transformer in PyTorch, train it on a free Colab GPU, and find out why a run diverged before you change anything.
Who it is for. Engineers who write production Python and now need the framework most lab postings name, and researchers who know the math but have not owned a training run.
The planned syllabus
Every module will meet the book's bar before it opens: the math derived on the page, a Colab notebook on Python 3.12, and measurements from real models.
- 01
Tensors and autograd
Shapes, broadcasting and the computation graph, checked against gradients you derive by hand.
- 02
A training loop from scratch
Data loading, the optimizer step, learning-rate schedules and checkpoints, with no trainer library hiding them.
- 03
A small transformer
Attention, the residual stream and layer norm, trained on a corpus small enough for Colab.
- 04
Debugging a run
Loss spikes, NaNs, dead activations and data bugs: read the curves and find the cause.
- 05
Mixed precision and memory
bf16, gradient accumulation and activation checkpointing, measured on the GPU you have.
- 06
Profiling
Where the step time goes, with the PyTorch profiler, and what to change first.
