Risk Tiers

This unlocks Model Evaluation itself — the parent concept you have notes on. Once you can sort use cases into risk tiers, you know how much evaluation a model needs: a harmless fun app gets light checks, a medical diagnosis tool gets rigorous ones. That lets you read any evaluation guide, because every metric, test set, and monitoring plan assumes you understand the stakes. It also leads directly to your neighbouring notes: Unlearning Verification (higher tiers demand proof of forgetting, not just claims), plus Four-Axis Quality, Offline vs Online Evaluation, and Regression Gates — all of which only make sense once you know which tier they apply to.

Think of the stakes first, the maths later. Not every AI system is equally dangerous. A model that suggests a recipe for leftover rice is harmless if it's wrong; a model that tells a doctor a dosage could kill if it's wrong. Risk tiers are just a way of saying: the more harm a mistake can do, the more checking we do before we trust it. You already do this in daily life. You don't test a torch before buying it — you just check it works. But you'd test brakes on a car carefully before driving. Risk tiers formalise that common sense: they assign a level of danger to a use case, and that level decides how much evaluation, oversight, and control we apply. The tier isn't a score of how good the model is; it's a measure of how bad failure would be. The mapping from harm to controls should be written down before anything goes wrong, because during an incident panic replaces judgement. It's like deciding your emergency exit route before a fire, not during it.

Imagine you're hiring a security guard for a building. The guard is a model; the building is the use case. A small shop needs a guard who can open the door and smile — a light check. A bank vault needs a guard with background checks, training, and supervision. You wouldn't spend the same time verifying both. Risk tiers are the security levels you assign based on what's inside the building. The tiers map to concrete controls: how many tests the guard must pass, how often they're audited, what happens if they fail. They don't just give you a colour on a chart — they tell you exactly what to do. This analogy stops working when you realise a model isn't a person: a guard can explain their decision, a model usually can't. And a model's behaviour can change with new data, unlike a guard's personality. But the core idea holds: the cost of a mistake determines the effort you put into preventing it. And you must agree on the tier before the guard starts work, because during a burglary is no time to decide how careful you should have been.

Risk tiers are a way of classifying AI use cases by the potential harm of a mistake, so that the depth of evaluation, the amount of human oversight, and the safety controls scale up with the stakes.

You're starting from zero, so I'll place this: Risk tiers aren't maths yet, but they're the foundation of Model Evaluation, which you have notes on. Think of Model Evaluation as 'measuring whether a model works' — and risk tiers tell you how carefully you must measure. A recipe suggester can get away with a rough accuracy check; a medical tool needs precise calibration and slice-based analysis. Your library also mentions Unlearning Verification, which is about proving a model has forgotten data. Risk tiers matter there because the verification required depends on the tier: a high-tier model must prove forgetting convincingly, a low-tier one can get away with a simpler test. So risk tiers tie into everything else — they're the lens through which you decide how much effort every other evaluation technique deserves.