Protected Attributes

This is the first step on the path to auditing AI systems for fairness and bias. Once you understand what protected attributes are and why they persist even when removed, you will be ready to study black-box vs white-box assessment — you cannot assess what you do not know how to identify — and perturbation robustness, where a tiny change to an input should not change an output if the system is stable. Later, you will learn to assign trust ratings to AI systems, and every rating rests on knowing which attributes a system must not use. Protected attributes are the anchor of that whole chain.

Think of a loan application form. It asks for your name, age, gender, address, income, and a few more details. Some of those details, like race, gender, age, or disability, are what we call protected attributes. In many countries, a bank is legally barred from using them to decide whether to give you a loan. But the bank still needs to decide. So it looks at other things: your income, your savings, your job history, where you live, even your postcode. Here is the catch: your postcode is not your race, but in practice it can be strongly related to race. If the bank uses your postcode, it might be indirectly using race anyway. The whole problem of fairness in AI is that the rule 'do not use race, gender, age, or disability' is easy to write but hard to follow, because the model can find indirect signals that stand in for those characteristics. Removing the attribute from the form does not remove bias — the bias is hidden in the remaining data. That is the single most important thing to understand here.

Imagine a detective taking a witness statement. The witness says things like 'the robber was a tall man in a mask.' But the detective also notices the witness is nervous, maybe because the robber is still nearby. The witness's fear is not the robber's attribute, but it might be a signal of the robber's presence. The detective cannot directly use the robber's race because he does not know it, but he can use the witness's reaction. In the same way, a hiring algorithm is told not to consider gender, but it can consider whether a person took a career break, which is more common among women. The career break is a proxy for gender. The detective's mistake would be to say 'I never use race, I only used fear' — but the fear points at the same thing. The algorithm's mistake is to say 'I never use gender, I only used career breaks' — but the break points at the same thing. This analogy works until it breaks: a detective is trying to catch one criminal, while an AI system applies the same rule to thousands or millions of people. A detective can override a wrong signal per case, but the AI applies the proxy automatically to everyone. That is why the problem is systematic, not occasional.

A protected attribute is a characteristic such as race, gender, age, or disability that law and ethics restrict from driving decisions made by AI or human decision-makers.

This concept sits at the root of Fairness and Bias, which you will study next. Your library notes already name the neighbouring topics: black-box vs white-box assessment, perturbation robustness, sentiment and emotion bias, and trust rating scales. All of these use the idea of a protected attribute. When you assess a system black-box, you look for disparities across groups defined by protected attributes. When you perturb an input to test robustness, you are checking whether a change that should not matter — like swapping a name — changes the output, which would indicate the model is using a proxy for a protected attribute. Sentiment bias is just a specific case of protected attributes leaking into model scores. And every trust rating you eventually assign will hinge on how well the system handles protected attributes. So this is not an isolated topic; it is the lens through which all fairness questions are seen.