Drift Detection

A model you train today will not keep working forever. The world changes: customers change their habits, prices shift, new products appear, old ones vanish. When that happens, your model's predictions slowly stop matching reality, and if you do not notice, the damage is silent — bad recommendations, missed fraud, wrong medical warnings — while everything looks fine on your dashboard. Learning drift detection gives you the alarm. It tells you the moment your model's assumptions start to break, so you can retrain, adjust, or investigate before the errors become expensive. This reason is why every serious AI operation — from a startup recommending movies to a bank approving loans — monitors for drift. It is one of the tools that separates a demo from a system people actually rely on. In your learning path, this unlocks the broader world of MLOps: monitoring, alerting, feedback loops, and the discipline of keeping models alive in production.

You are a doctor who has been seeing the same kind of patient for years. You are excellent at diagnosing the common cold. But then a new virus arrives, or the climate changes and allergies become more frequent. Your usual rules stop working as well. You would want to notice, early, that something is different — before a patient's illness gets worse. Drift detection is like the equipment that tells the doctor, 'The pattern of symptoms you are seeing is no longer what you are used to.' In AI, a model is trained on a particular slice of the world. It learns patterns from that training data. When the real world's data starts to look different from that training data, the model's predictions become less reliable. Drift detection measures how different — and alerts you. There are three kinds of things that can drift: the input data itself (for example, your customers suddenly earn less), the predictions the model makes (it starts approving far more loans than before), and the relationship between input and outcome (the rules that say 'high income → repays loan' stop being true). Drift detection watches one or more of these and raises a flag when the change is too big to ignore. It gives you a signal, not a judgment. A drift alert does not say 'your model is broken' — it says 'the world looks different from what you trained on; you should look closer'.

Think of yourself as the caretaker of a large public garden. You have planted many flowers, and you water them according to a carefully learned schedule: each type of flower needs a certain amount of water each day, based on your observations of past seasons. You have a smart watering system that you trained with all your experience. But one year, the climate shifts subtly. Rain falls less often, the sun is stronger, and some flowers start to wilt. The watering system, though, still follows its old rules. It does not know the weather has changed. If you were the gardener, you would walk through the garden each morning, look at the flowers, and notice they are drooping earlier than usual. That is the alarm. Drift detection is your walk through the garden. It compares what the system expects (based on the training data) with what it is currently seeing (the live data coming in). If the leaves are drooping — if the incoming data is significantly different — it rings a bell. The bell does not tell you what to change; it just tells you to pay attention. You still need to inspect the garden, figure out whether the soil is dry, whether a disease is spreading, or whether it is just autumn and the flowers are supposed to wilt. Drift detection is the same. It flags the difference; you, the human, diagnose the cause. The analogy stops working, though, because a garden is a simple thing — there is only one kind of 'expected' and 'actual.' In AI, the data is multidimensional, and drift can be subtle and temporary. A weather change that affects all flowers (global drift) is easy to spot, but a disease that only affects one type of rose (a small subset of the data) might be missed unless you look at each flower species separately. So drift detection needs to watch the whole garden and each bed individually — and it needs to tolerate natural seasonal variation, or it will cry wolf.

Drift detection is the systematic monitoring of a model's incoming data and predictions versus its training-time expectations, to flag when the input distribution, prediction distribution, or the input-output relationship has changed enough that the model's performance may be degrading.

This concept sits inside MLOps, the practice of keeping machine learning models working in production. You already have notes on model deployment, CI/CD, and model evaluation — those cover how a model gets built and shipped. Drift detection is what comes after: monitoring the model once it is live. It is the 'check the engine while it runs' step. It connects directly to several neighbouring topics in your library: Model Registry and Versioning (when drift is detected, you may need to roll back to an older model version or promote a retrained one), Canary Releases (a canary release compares a new model's metrics against the old one — drift detection can inform when to expand or roll back), Human-in-the-Loop Feedback (drift alerts often trigger human review of predictions to decide whether to retrain), and Production Feedback Loops (drift detection is the signal that closes the loop from production back into the next training run). In short, drift detection is the bridge from building a model to keeping it alive.