SISA Sharded Training

This concept unlocks understanding of how modern AI systems can delete your data from a trained model — a legal requirement under the Right to Be Forgotten. By learning SISA, you'll be able to grasp the rest of the Machine Unlearning map: Exact Unlearning (full retraining is the gold standard but expensive), Approximate Unlearning (fast but not guaranteed), Federated Unlearning (the distributed version), and how to verify that deletion worked (Unlearning Verification). You'll also understand the central tension of Utility Preservation — how to forget without wrecking the model. This is the foundation for building responsible AI systems that comply with data protection regulations.

Imagine you've baked a cake. If someone tells you there was a bad egg in one batch of ingredients, you can't just pick the egg out — you'd have to bake a whole new cake without it. That's the problem with retraining a machine learning model from scratch: it's expensive and slow. SISA solves this by baking many small cakes instead of one big one. You split your data into separate shards, train a separate model on each shard, and then combine their votes to get a final answer. When a deletion request comes, you only need to rebake the one cake that used the bad egg — the one shard's model that saw that record. The other shards are untouched. The catch is that each small cake is a bit more crumbly — the model trained on a smaller portion of data might be slightly less accurate than one trained on everything. That's the trade-off you accept for being able to delete cheaply.

Think of a committee making decisions. Instead of one all-knowing oracle, you have several experts, each trained on a different slice of the evidence. When a new case comes, each expert gives their opinion, and the committee votes (aggregates). If one expert was given a piece of evidence that later must be forgotten, you don't need to retrain the whole committee — you just replace that one expert with a fresh one trained on the remaining evidence. The other experts are unaffected because they never saw the deleted evidence. This works because each piece of evidence is assigned to exactly one expert (sharded and isolated). But here's where the analogy breaks: a committee of experts is usually better than one expert who saw everything, but in machine learning, training on less data often makes each model worse. A model trained on 10,000 records is usually more accurate than one trained on 2,000. So SISA sacrifices a little accuracy to gain the ability to delete efficiently. Also, the voting isn't just majority — it's often averaging the outputs, so if one model gives a wild answer, it's averaged out, but not if it's a critical part of the decision.

SISA (Sharded, Isolated, Sliced, and Aggregated) training is a method of training machine learning models where the data is split into disjoint shards, each shard trains its own separate model independently, and the final prediction is an aggregate of all the shard models' outputs, such that a data deletion request only requires retraining the single shard that has seen the deleted record.

This concept builds on nothing yet, but it connects to several neighbouring topics you'll encounter: Exact Unlearning (the goal of perfect removal — SISA makes this affordable), Approximate Unlearning (a faster but weaker alternative), Federated Unlearning (similar idea but with updates instead of raw data), Deletion Requests (the operational process), Right to Be Forgotten (the legal driver), and Utility Preservation (the trade-off SISA introduces). Your library notes, from the book *Machine Unlearning: Concepts, Techniques and Applications*, stress that SISA makes exact unlearning tractable by bounding how much of the model any one record touched.