Lesson 20. Reading a Loss Curve

Mission link: "Can read a training dashboard and say whether the run is on track" is the Success looks like bullet this stage serves. This lesson covers what to actually watch on a live dashboard; Lesson 21 covers evaluating saved checkpoints against held-out data.
Primary source: Paper: "Scaling Language Models: Methods, Analysis & Insights from Training Gopher", Rae et al., 2021
Prerequisites: Lesson 19

Warm-up

1. ▢ What did Zhang et al. notice was correlated with loss divergence during OPT-175B's training, besides the loss curve itself?

2. ▢ Why does recovering from a loss divergence argue for keeping more than just the single most recent checkpoint?

Know this

The training loss curve, and what "healthy" means for it

The most basic thing a training dashboard shows is the training loss itself, plotted against training step or token count. Stage 3's scaling laws give a real expectation for its shape: loss should fall off smoothly, roughly following a power-law-like curve as compute and tokens accumulate, not in a straight line and not in a series of unrelated jumps. A curve that tracks this general shape is behaving as expected; a curve that flattens out far earlier than the scaling-law trajectory would predict, or one that shows a sudden sharp rise (a loss spike, Lesson 16), is a training loss curve reporting something worth investigating, before assuming the run is simply on track.

Two signals that catch trouble before the loss curve clearly shows it

Watching training loss alone is not enough, because by the time a spike is visible in the loss itself, the underlying problem may already have been building for a while. Zhang et al.'s account of OPT-175B gives two other signals that gave earlier warning of the same trouble the loss curve eventually showed: their dynamic loss scalar, part of the mixed-precision setup, crashing toward zero, and the norm of the final layer's activations spiking upward. Both are symptoms of a training run heading toward instability, visible before, or alongside, the loss itself clearly diverging, which is exactly why Zhang et al. used them, not just the loss curve, to judge whether a given point in training was a safe one to restart from.

The gradient norm, watched over time

Lesson 16 covered gradient-norm clipping as a per-step safeguard, bounding how large any single update can be. The same gradient norm, watched as a curve over the course of training rather than only checked against the clipping threshold at each step, is itself a useful signal: a gradient norm that is trending upward over time, or spiking repeatedly even after clipping, can be an early sign of the same kind of instability a loss spike would eventually show more dramatically, giving a chance to intervene, such as by lowering the learning rate, before a full divergence forces a restart.

Watching, continuously

None of these signals help if nobody, or nothing automated, is actually looking at them. Stage 8 established that failures and instabilities are a routine, frequent part of a large training run, not a rare event; a dashboard showing loss, gradient norm, and a numeric-stability signal like a loss scalar is only useful if it is checked often enough, or alerts automatically enough, to catch a developing problem before it costs far more progress than an earlier catch would have.

Practice

1. ▢ A team's training loss curve is decreasing, but noticeably more slowly than Stage 3's scaling-law relationships would predict for their model size and token count so far. Is this necessarily a sign of a problem, or could it be consistent with a healthy run?

2. ▢ Why might a loss scalar crashing toward zero, or an activation norm spiking, give earlier warning of instability than the training loss curve itself?

3. ▢ Which of these best describes why a dashboard tracking loss, gradient norm, and a stability signal like a loss scalar is only useful in practice if it is monitored frequently?

   • a) These metrics are only meaningful once per training run, at the very end
   • b) Failures and instabilities are frequent, routine events at scale (Stage 8), so a problem caught late costs far more lost progress than the same problem caught early
   • c) The metrics themselves change their meaning if checked too often
   • d) Frequent monitoring is only useful for very small models

Real-world reps

• [ ] Find a training framework's or experiment-tracking tool's dashboard (Weights & Biases, TensorBoard, or similar) and identify which of this lesson's signals (loss, gradient norm, a numeric-stability indicator) it surfaces by default, and which would need to be added explicitly.
• [ ] Read the paragraph in the OPT paper (linked above) connecting loss divergence to the loss scalar and activation norm, and write one sentence on which signal you would check first if you saw a loss spike on a dashboard with all three available.
• [ ] Tomorrow: without looking ahead, write a one-sentence prediction for Lesson 21: if training loss alone does not tell you how well a model performs on the tasks people will actually use it for, what would need to be measured differently to check that during training, rather than only after it finishes?

Going further

• Paper: "Scaling Language Models: Methods, Analysis & Insights from Training Gopher", Rae et al., 2021
• Paper: "OPT: Open Pre-trained Transformer Language Models", Zhang et al., 2022
• Resources

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.