Skip to content

Course in preparation · Coming soon

PyTorch for Research Engineers

Write the training loop a lab team would trust. Build tensors, autograd and a small transformer in PyTorch, train it on a free Colab GPU, and find out why a run diverged before you change anything.

Who it is for. Engineers who write production Python and now need the framework most lab postings name, and researchers who know the math but have not owned a training run.

The planned syllabus

Planned lessons will help you understand each method, apply it to an AI problem, and judge whether the result holds up.

  1. 01

    Tensors and autograd

    Shapes, broadcasting and the computation graph, checked against gradients you derive by hand.

  2. 02

    A training loop from scratch

    Data loading, the optimizer step, learning-rate schedules and checkpoints, with no trainer library hiding them.

  3. 03

    A small transformer

    Attention, the residual stream and layer norm, trained on a corpus small enough for Colab.

  4. 04

    Debugging a run

    Loss spikes, NaNs, dead activations and data bugs: read the curves and find the cause.

  5. 05

    Mixed precision and memory

    bf16, gradient accumulation and activation checkpointing, measured on the GPU you have.

  6. 06

    Profiling

    Where the step time goes, with the PyTorch profiler, and what to change first.

Start now with what it builds on

All courses