Federated Unlearning

This concept sits at the meeting point of two ideas you have already met in your library: Machine Unlearning (removing a record's influence from a trained model) and Federated Learning (training a model across many devices without ever collecting the data centrally). Mastering federated unlearning is what lets you honour deletion requests in real products — a user asks to be forgotten, and the model must actually forget them even though their data never left their phone. It unlocks practical work in Data Privacy Compliance and AI Governance and Regulation, because regulators increasingly ask: 'If a citizen demands erasure, can your model truly comply?' Without federated unlearning, the answer is a legal and technical no.

Imagine a school where each student keeps their own diary, and the teacher never sees the diaries. At the end of each week, the teacher asks every student to write a short 'summary note' — a few lines capturing what they learned — and hands those notes to the teacher. The teacher blends all the notes into a class summary, which becomes the next week's starting point. The teacher never reads the diaries, only the summaries. That is federated learning: each phone or device trains on its own data, sends only a small 'update' (the summary), and the central server averages the updates into a better model.

Now suppose a student leaves the class and says, 'Please remove everything I contributed from the class summary.' The teacher cannot go back to the diary — they never had it. They only have the summary notes they received. So how do they remove one student's influence from the class summary? That is federated unlearning: removing a participant's data influence from a federated model, without ever having had the raw data centrally.

The central difficulty is that the teacher cannot simply 'delete' a summary from the blended average, because each week's summary built on top of previous summaries. The student's influence has soaked into the whole class model over time. It is like trying to remove a drop of ink from a glass of water — you can't pick it out, you can only dilute or start over.

Think of a pot of soup being stirred by several cooks, each adding a spoonful of their own spice. No cook ever hands over their entire recipe — only a spoonful that gets mixed in. After many spoonfuls, the soup tastes like a blend of everyone. Now one cook says, 'I want my spoonful out — I don't want my spice in the soup.' That is the federated unlearning problem. You cannot reach into the pot and pull out one spoonful, because it has already diffused and mingled.

The only way to be absolutely certain the requested spoonful is gone is to throw out the whole pot and start again, asking all the cooks to re-add their spoonfuls except the one who left. That is full retraining — the gold standard for correctness, but very expensive.

To make it cheaper, imagine the soup is cooked in several separate small pots (one per shard). Each cook only adds their spice to one pot. When a cook leaves, you only throw out that one pot and re-cook it with the remaining cooks — the other pots are untouched. That is SISA-style sharded training, which you have in your notes as SISA Sharded Training.

But if the soup is already one big pot, and you cannot re-cook it right now, you can try a shortcut: stir in some neutral ingredient to 'dilute' the unwanted flavour. That is approximate unlearning — it reduces the influence but does not guarantee it is completely gone. In federated unlearning, you often have to use such shortcuts because you cannot retrain the whole federation quickly.

Federated unlearning is the process of removing the influence of one participant's data from a machine learning model that was trained across many devices, without ever having access to that participant's raw data, and doing so efficiently enough to be practical.

This concept builds directly on your notes on Machine Unlearning, SISA Sharded Training, Approximate Unlearning, and Exact Unlearning. The parent idea is that after training, you want to undo the effect of certain data. Federated unlearning adds the twist that you never had the data centrally, so you cannot simply 're-train without that row' — you have to work with the updates that were sent. It also connects to Deletion Requests and the Right to Be Forgotten, because the legal obligation to erase data extends to models, and federated unlearning is the technical mechanism that makes that possible in a federated setting. In your library, the book *Machine Unlearning: Concepts, Techniques and Applications* notes that naive retraining is both the correctness baseline and the cost benchmark — that applies here too, but retraining a federated model is especially costly because it involves contacting all clients again.