Course in preparation · Coming soon
Post-Training: From Fine-Tuning to Reinforcement Learning
Take a small open model through supervised fine-tuning, LoRA, preference optimization and reinforcement learning with verifiable rewards. Derive each objective, run it, and measure what it changed.
Who it is for. Applied-AI and research candidates who have used post-trained models and now need to explain, run and evaluate post-training themselves.
The planned syllabus
Planned lessons will help you understand each method, apply it to an AI problem, and judge whether the result holds up.
- 01
What post-training changes
Derive the next-token loss restricted to the assistant's tokens, and measure how far a base model is from following an instruction.
- 02
Supervised fine-tuning
Chat templates, loss masking and data formatting; fine-tune a small model and measure held-out loss and task accuracy.
- 03
LoRA and parameter-efficient tuning
Derive the low-rank update and its parameter count; measure what rank buys and what it costs.
- 04
Preferences and reward models
Derive the Bradley-Terry model, train a reward model, and measure its agreement on held-out preferences.
- 05
Direct preference optimization
Derive DPO from the KL-regularized objective, run it, and compare it with fine-tuning alone on a paired evaluation.
- 06
Reinforcement learning with verifiable rewards
Policy gradients, advantages and group baselines, run on problems whose answers can be checked; measure pass@k.
- 07
Evaluating a post-trained model
Contamination, regressions on held-out skills and error bars, written up the way a lab would.
