Catastrophic Forgetting

Catastrophic forgetting is the single reason most attempts to adapt a pretrained model to a new domain quietly ruin it. If you fine-tune an LLM on your company's support tickets for a week and it starts forgetting how to answer general questions, you have hit this wall. Understanding it tells you when full fine-tuning is a mistake, why methods like LoRA (which you will study next) exist, and why mixing general data into your training set is the cheapest insurance you will ever buy. It is the concept that connects fine-tuning to the everyday decision every ML engineer makes: do I adjust the whole model, or do I bolt on a small adapter? Master this, and you will never be the person who wonders why their model got stupider after 'improving' it.

Imagine you have a brilliant generalist employee who knows a little about everything — customer service, legal, finance, cooking. Now you want them to specialise in your company's loan approval process. You send them on a two-week intensive training course, where all they see, all day, is loan applications: hundreds of thousands of them. By the end, they are a loan-approval expert. But when you casually ask them a question about cooking, they hesitate. Ask them about the legal basics they used to know, and they give a shaky answer. The intensive training overwrote some of what they knew. It did not happen because the course was bad — it happened because the course was so focused that the old knowledge had no chance to be practised, and the brain (or the model) kept only what it saw recently. Catastrophic forgetting is exactly this, in a neural network. The model's knowledge lives in its weights — millions of tiny numbers. When you train on new data, you nudge those numbers to fit the new examples. Every nudge that helps with a loan application is a nudge away from what the model knew about cooking. If the new data is wildly different from the old, the nudges are large, and the old knowledge is destroyed. It is not gradual decay; it is a cliff. One epoch of aggressive new data and the general ability collapses.

Think of a musician who has spent years mastering classical piano. The repertoire is vast — Bach, Beethoven, Chopin — all stored in muscle memory. Now they decide to learn jazz. They hire a jazz coach who drills them for weeks, eight hours a day, on nothing but syncopated chords and blues scales. Jazz is hard and demands new finger patterns that fight the classical habits. After a month of this, they sit down to play a Chopin nocturne and find their fingers keep sliding into jazz phrasings. The classical pieces are not gone entirely, but they are badly degraded, because the intense jazz practice rewired the same neural pathways. The analogy breaks down in two places, though. First, a human musician can deliberately rehearse old pieces to preserve them; a neural network has no such intention — it simply responds to the data it is fed. Second, a human's decline is gradual, whereas for a neural network the loss can be sudden and near-total if the new data is very different. The remedy in both cases is the same: keep practising the old material alongside the new. That is exactly the standard mitigation for catastrophic forgetting in AI: mix general data into your fine-tuning set.

Catastrophic forgetting is the phenomenon where a neural network, while being trained on new data for a specific task, loses capabilities it previously had on older, broader data, because the weight updates that fit the new data overwrite the weights that encoded the old knowledge.

You are studying this as part of fine-tuning, which is the parent concept in your library. Fine-tuning means taking a pretrained model and continuing training on new data. Catastrophic forgetting is the central risk of that process, as noted in your library's source, Fine-Tuning with Python. You have already learned about pretraining (how the model got its general knowledge) and large language models. This concept connects directly to LoRA and QLoRA, which you will study next: those methods freeze the base weights and train small adapters, precisely to avoid catastrophic forgetting. It also connects to continued pre-training and instruction tuning, where the risk is real but the data is often close enough to the original that the damage is smaller. Your library notes that 'mixing in general data during tuning is the standard mitigation' — you will see exactly how to do that in the worked example.