Lesson 24. The Regression Suite

Mission link: "Build an eval that catches a regression before shipping" is the mission's own wording. This is that lesson.
Primary source: Paper: "An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning", Luo et al., arXiv:2308.08747
Prerequisites: Lesson 23, Lesson 22

Warm-up

1. ▢ Why is loss the wrong basis for a decision?

2. ▢ Two documented biases of model judges, and their mitigations?

3. ▢ Why is your task's held-out set blind to catastrophic forgetting?

Know this

The failure this exists to catch

Fine-tuning moves weights toward your task. Abilities that were not reinforced can degrade, and this is catastrophic forgetting. The empirical literature finds it is not a rare edge case in continual fine-tuning of language models: general capabilities including instruction following measurably decline, and the effect can grow with model scale rather than shrinking.

The dangerous property is that every number you are already looking at goes the right way. Training loss falls, your held-out task metric improves, your probes look good, and the model has become worse at things you never measured. You ship, and the reports come from a direction your evaluation could not see.

A held-out set measures whether you learned the task. A regression suite measures whether you broke anything else. They are different instruments and you need both.

What goes in it

Four categories. All four, not a selection.

1. General instruction following. Can it still follow an ordinary instruction with no relation to your task? Summarise something, answer a general question, follow a multi-step request.

2. Format and behavioural compliance. Does it still respect a system prompt? Still produce valid JSON when asked? Still stop when it should? A model trained on one rigid format sometimes loses the ability to produce any other.

3. Safety and refusal behaviour. If the base model refused certain requests, does the fine-tune still refuse them? Fine-tuning on benign data can degrade safety alignment as a side effect. That is a documented and repeatedly reproduced result, not a hypothetical. If you inherited alignment behaviour from an instruct model, you inherited a responsibility to check it survived.

4. Adjacent capability. Things near your task that you did not train. Fine-tuned on English support tickets? Check another language. Fine-tuned on one document type? Check another.

Building it

Small, fixed, and read by a human.

regression/
├── instruction-following.jsonl   # 20 general instructions
├── format-compliance.jsonl       # 20 schema and system-prompt cases
├── safety.jsonl                  # 20 cases the base refused
├── adjacent.jsonl                # 20 near-task, untrained cases
└── baseline/                     # base model outputs, committed

Eighty examples total. That is deliberately small: enough to catch a real regression, small enough that a person actually reads the diffs.

The critical, frequently-omitted step: record the base model's outputs before you train, and commit them. A regression is a change, so you need the before. Without a stored baseline you are left comparing against your memory of how the model used to behave, which is not evidence.

# Run once, before any training. Commit the result.
for case in regression_cases:
    out = base_model.generate(**tok(case["prompt"]), do_sample=False, max_new_tokens=256)
    baseline[case["id"]] = tok.decode(out[0], skip_special_tokens=True)

Greedy decoding, so a diff means a weight change rather than a sampler roll.

Reading it

Run the suite on every candidate checkpoint. Diff against the baseline. Then classify each change:

The third row is the whole point. One clear regression in safety or instruction following outweighs a couple of points on your task metric, because the failure mode is unbounded and the gain is not.

Reducing forgetting, when you find it

Options roughly in order of what they cost you:

1. Fewer steps, or an earlier checkpoint. Forgetting grows with training. Often the cheapest fix, and it usually costs little task performance.
2. Lower learning rate. Smaller updates disturb less.
3. Lower rank, or fewer target modules. Less capacity to overwrite with.
4. Mix in general instruction data. A few percent of general instruction-following examples alongside your task data. Effective and well-established, and it costs you dataset purity rather than model quality.
5. Keep the adapter unmerged and route. The most complete answer: apply the adapter only for requests that need it, and serve the untouched base for everything else. Forgetting becomes irrelevant because the base is still there. Lesson 25.

Option 5 is the one people overlook, and it is often the correct architecture. If your fine-tune serves one narrow purpose inside a larger product, you do not need one model that does everything.

Automate it

The suite should run on every checkpoint, unprompted, and refuse to pass on a regression. An evaluation that depends on someone remembering to run it will be skipped on the run where it mattered, the one under deadline pressure.

Practice

1. ▢ Why does a held-out task metric that improves monotonically fail to rule out a regression?

2. ▢ Name the four regression categories.

3. ▢ You have a regression suite but no stored baseline. What can you conclude from running it?

4. ▢ Task metric up 3 points; the model no longer refuses a request the base refused. Ship?

5. ▢ Which most completely eliminates the forgetting problem?

   • a) Mixing general instruction data into the training set
   • b) Serving the base model unmerged and routing by request
   • c) Lowering the learning rate and training for fewer steps
   • d) Reducing the adapter rank and the target module count

6. ▢ Why greedy decoding for the regression suite?

Real-world reps

• [ ] Write 20 regression cases, five in each category. Include at least one the base model refuses.
• [ ] Generate and commit base-model baseline outputs, greedily, before any training.
• [ ] Run the suite on your existing adapter and diff. Classify every change into the four verdicts.
• [ ] Tomorrow: wire the suite into your training script so it runs on every checkpoint without being asked.

Going further

• Paper: "An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning", Luo et al., arXiv:2308.08747
• Paper: "Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!", Qi et al., arXiv:2310.03693
• Failure modes
• Lesson 25. Serving Adapters

Not landing? Reread the primary source at the top, since this lesson compresses it and compression is where understanding leaks. Check the glossary for any term that felt slippery.

If the lesson itself is unclear rather than the material, that is a defect: open an issue.