Decision Trees

Decision trees let you make predictions by asking a simple series of yes-or-no questions, much like a flowchart you might draw on a whiteboard. This is valuable because the logic behind every prediction is completely transparent: you can trace exactly which features led to a specific outcome. This clarity makes them indispensable when you need to explain your decisions to stakeholders—for example, in credit scoring, medical diagnosis, or fraud detection, where 'why' matters as much as 'what.' They are particularly useful for structured data like spreadsheets, and they serve as the fundamental building block for more advanced ensemble methods that combine many such trees to improve accuracy.

Imagine you have a list of customers, each described by a few numbers: their age, their monthly spending, and whether they have a loyalty card. You want to predict whether they will churn — that is, whether they will stop using your service within the next month. A decision tree asks a series of questions: 'Is their age under 30?' If yes, go down one branch; if no, go down another. Then on each branch, it asks another question: 'Is their monthly spending under ₹2000?' And so on, until it reaches a final node — a terminal box that says 'likely to churn' or 'likely to stay.' The tree is built from historical data: it looks at past customers whose churn is known, and it learns the best questions to ask and the best order to ask them in. The beauty is that the final model is a set of rules you can read and understand: 'If age under 30, and spending under ₹2000, then 80% of such customers churned.' No other model type gives you this clarity.

Think of a doctor diagnosing a patient. The doctor does not consider all possible diseases at once. Instead, she follows a flowchart: 'Is the temperature above 38°C?' If yes, 'Is there a cough?' If yes, 'Is the throat red?' — and so on. Each answer narrows the possibilities until she reaches a diagnosis. A decision tree works exactly the same way. The data is the patient's symptoms (features), the questions are the tests (thresholds on features), and the final diagnosis is the prediction. The doctor's flowchart is learned from years of experience; the decision tree's flowchart is learned from historical data. A caveat: a real doctor can adjust their questions dynamically based on new information, whereas a trained decision tree is fixed once built and cannot adapt to individual nuances outside its original training data.

A decision tree is a predictive model that divides data into groups by repeatedly asking single-feature questions, creating a readable map of if-then rules that leads from the initial question to a final prediction.

This concept introduces two foundational ideas: splitting criteria, which are the mathematical methods used to decide which question best separates the data at each step, and model complexity, which refers to how detailed the tree becomes. These concepts help you understand how decision trees relate to ensemble methods like Random Forests, which combine many trees to reduce error, and other classifiers like K-Nearest Neighbors.