SISA Sharded Training

This concept unlocks understanding of how modern AI systems can delete your data from a trained model — a legal requirement under the Right to Be Forgotten. By learning this approach, you'll be able to grasp the rest of the Machine Unlearning map: Exact Unlearning (full retraining is the gold standard but expensive), Approximate Unlearning (fast but not guaranteed), Federated Unlearning (the distributed version), and how to verify that deletion worked (Unlearning Verification). You'll also understand the central tension of Utility Preservation — how to forget without wrecking the model. This is the foundation for building responsible AI systems that comply with data protection regulations.

Imagine you've baked a cake. If someone tells you there was a bad egg in one batch of ingredients, you can't just pick the egg out — you'd have to bake a whole new cake without it. That's the problem with retraining a machine learning model from scratch: it's expensive and slow. This method solves this by baking many small cakes instead of one big one. You split your data into separate groups, train a separate model on each group, and then combine their votes to get a final answer. When a deletion request comes, you only need to rebake the one cake that used the bad egg — the one group's model that saw that record. The other groups are untouched. The catch is that each small cake is a bit more crumbly — the model trained on a smaller portion of data might be slightly less accurate than one trained on everything. That's the trade-off you accept for being able to delete cheaply.

Think of a committee making decisions. Instead of one all-knowing oracle, you have several experts, each trained on a different slice of the evidence. When a new case comes, each expert gives their opinion, and the committee votes (combines them). If one expert was given a piece of evidence that later must be forgotten, you don't need to retrain the whole committee — you just replace that one expert with a fresh one trained on the remaining evidence. The other experts are unaffected because they never saw the deleted evidence. This works because each piece of evidence is assigned to exactly one expert (divided into separate, isolated parts). But here's where the analogy breaks: a committee of experts is usually better than one expert who saw everything, but in machine learning, training on less data often makes each model worse. A model trained on 10,000 records is usually more accurate than one trained on 2,000. So this method sacrifices a little accuracy to gain the ability to delete efficiently. Also, the voting isn't just majority — it's often averaging the outputs, so if one model gives a wild answer, it's averaged out, but not if it's a critical part of the decision.

This training method involves splitting data into separate parts, training independent models on each part, and combining their outputs for final predictions, such that deleting a record only requires retraining the single part that contained it.

This concept builds on nothing yet, but it connects to several neighbouring topics you'll encounter: Exact Unlearning (the goal of perfect removal — this method makes this affordable), Approximate Unlearning (a faster but weaker alternative), Federated Unlearning (similar idea but with updates instead of raw data), Deletion Requests (the operational process), Right to Be Forgotten (the legal driver), and Utility Preservation (the trade-off this method introduces). Your library notes, from the book *Machine Unlearning: Concepts, Techniques and Applications*, stress that this method makes exact unlearning tractable by bounding how much of the model any one record touched.