Counterfactual Explanations

This skill lets you answer the most common question people have when an automated system says 'no': 'What would I need to change to get a different result?' You will use this when explaining why a loan was rejected, why a medical test flagged a risk, or why a fraud alert fired. Instead of just receiving a verdict, the person affected gets a clear path to a positive outcome—such as earning ₹15,000 more per month or reducing debt by ₹2,00,000. Without this, you can build and deploy a model, but you cannot explain it to the person affected in a way they can act on—and that is where trust is won or lost.

You are at a bank counter. The loan officer says: 'Your application is rejected.' You ask, 'Why?' The officer could answer in many ways: 'Your debt-to-income ratio is 0.6, which is above our limit,' or 'The model gave you a 0.34 score.' But the most useful answer for you is: 'If your monthly income were ₹15,000 higher, or your existing debt were ₹2,00,000 lower, we would have approved you.' That is a counterfactual explanation. It tells you the *smallest change* to your situation that would have flipped the decision from 'no' to 'yes'. It is action-oriented: you can do something with it. You know exactly what to aim for.

Think of driving to an interview. You arrive late, and the interviewer says, 'You missed the time slot.' But a helpful colleague takes you aside: 'If you had left home 20 minutes earlier, you would have made it.' That is a counterfactual: a small change (departure time) that would have changed the outcome (being on time). Now, imagine the model’s decision depends on factors like income, debt, credit history, age, and location. A counterfactual explanation is the colleague saying, 'If income went up by ₹10,000, the decision flips from rejected to approved.' The analogy breaks here: in driving, you usually control the change (you can leave earlier). For a model, some changes may be impossible (you cannot change your age) or unethical. So a good counterfactual must be *feasible* — a change you can actually make in the real world, not just a number that works in a formula.

A counterfactual explanation identifies the minimal adjustment to the input data that would result in a different outcome from a decision-making system. It specifies what could have been done differently to achieve a desired result.

This concept sits beside Activation Patching and Model Interpretability. Activation Patching is a method for inspecting a model by swapping out its internal calculations during processing, while counterfactual explanations look only at the input and output, ignoring how the calculation happens inside. Both help you understand how decisions are made: one probes the internal machinery, and the other explores the boundaries of possible inputs.