Trust Rating Scales

You're here because you want to know which AI system to trust — whether you're choosing a vendor, explaining a model to a regulator, or evaluating a system you built yourself. After this page, you'll be able to read any trust rating (like an AI score out of 10) and ask the questions that expose whether it's real or marketing. You'll understand why a rating earned last year may be worthless today, and you'll have the foundation for the next topics in your library: Perturbation Robustness (how to test a system's stability), Black-Box vs White-Box Assessment (how ratings are actually produced), and ultimately Fairness and Bias (where ratings often hide unfairness).

Think of trust ratings like credit scores, but for AI systems. When you get a credit score from a bank, it's a single number that summarizes how likely you are to repay a loan. But that number is only useful because the bank tells you how it was calculated — your payment history, debt, length of credit, and so on. If the bank just said 'your score is 720' without explaining the method, you couldn't compare it to another bank's score, and you couldn't know what to improve. A trust rating for an AI system is the same: a graded score that expresses how trustworthy the system is, but the score is meaningless unless the methodology for producing it is published. The whole point of a scale is comparability — you want to say 'this system scores 8 out of 10, and that one scores 6, so I'll choose the first.' But if each vendor uses a different scale, or no scale at all, you can't compare them. And here's the catch: a rating is a snapshot in time. As the model's data changes (maybe the training data gets updated, or the real world shifts), the rating must be re-earned. A system that was trustworthy last month might be dangerous now, and (unlike a credit score, which updates monthly) an AI trust rating is only valid until the next update or the next deployment. So a trust rating is not a permanent stamp of approval; it's a living measure that has to be rechecked whenever the system or its context changes. This intuition is the core: comparable, methodical, and time-limited.

Imagine you're hiring a chef for a restaurant. You could just taste one dish and decide, but that's unreliable — the chef might be great at pasta and terrible at desserts. So you design a rating system: you ask the chef to prepare a set menu of ten dishes, you score each on taste, presentation, and hygiene, using a rubric that says '5 = excellent, 1 = poor.' Now you have a comparable score for each chef. But here's where the analogy breaks: a chef's skill is fairly stable — once they're good, they stay good for years. An AI system is not like that. Its behavior can change when you update the data it learns from, or when the world changes (for example, a fraud-detection model trained on pre-pandemic data might be useless now). So the trust rating for an AI system must be re-earned much more often than a chef's rating. Also, a chef is a white box — you can watch them cook and see what ingredients they use. An AI system is often a black box — you can only see the dish (the output) and not the recipe (the weights). So the rating methodology has to work from the outside, by poking and prodding: you order the same dish with slight variations and see if it comes out consistent. That's perturbation (which you'll learn about next). And unlike a chef, the AI system's 'taste' might be biased against certain customers without you being able to see why — so the rating needs to include fairness checks, not just quality. So the chef analogy gets you the idea of a rubric, but it stops working when you push on time-sensitivity and black-box access.

A trust rating scale is a published, graded scoring system that expresses how trustworthy an AI system is, in a way that allows comparison across systems, and that must be updated as the system or its data changes.

You're learning this as part of the Fairness and Bias concept in your library. Neighbouring topics you've already noted are Black-Box vs White-Box Assessment (ratings are often produced black-box), Perturbation Robustness (how stability is measured), and Protected Attributes (ratings must not conflate statistical bias with confounding bias). Trust ratings sit on top of those: they aggregate findings from perturbation tests and fairness checks into a single comparable score.