Decision Trees

Decision trees are the gateway to understanding how machine learning models actually make decisions. They are the most interpretable of all models — every prediction can be traced back to a simple chain of 'if-then' rules. This makes them the model of choice for any situation where you must explain your decisions to a human: credit scoring, medical diagnosis, fraud alerts, or regulatory compliance. They also form the building block of far more powerful methods: random forests, gradient boosting, and XGBoost are all "many decision trees working together." By mastering decision trees, you will understand the core idea of splitting data on rules, and you will be ready to learn how ensembles correct a tree's weaknesses. They also appear throughout practical machine learning for small, tabular data — the default choice when your data is a spreadsheet rather than images or text. This concept unlocks: Random Forests, Gradient Boosting, Model Interpretability, and Feature Engineering.

Imagine you have a list of customers, each described by a few numbers: their age, their monthly spending, and whether they have a loyalty card. You want to predict whether they will churn — that is, whether they will stop using your service within the next month. A decision tree asks a series of questions: 'Is their age under 30?' If yes, go down one branch; if no, go down another. Then on each branch, it asks another question: 'Is their monthly spending under ₹2000?' And so on, until it reaches a leaf — a final box that says 'likely to churn' or 'likely to stay.' The tree is built from historical data: it looks at past customers whose churn is known, and it learns the best questions to ask and the best order to ask them in. The beauty is that the final model is a set of rules you can read and understand: 'If age under 30, and spending under ₹2000, then 80% of such customers churned.' No other model type gives you this clarity.

Think of a doctor diagnosing a patient. The doctor does not consider all possible diseases at once. Instead, she follows a flowchart: 'Is the temperature above 38°C?' If yes, 'Is there a cough?' If yes, 'Is the throat red?' — and so on. Each answer narrows the possibilities until she reaches a diagnosis. A decision tree works exactly the same way. The data is the patient's symptoms (features), the questions are the tests (thresholds on features), and the final diagnosis is the prediction. The doctor's flowchart is learned from years of experience; the decision tree's flowchart is learned from historical data. But the analogy has a limit. A doctor can ask a test and get a result instantly, and can adjust her questions based on the patient's responses. A decision tree is built once, from a fixed dataset, and then used as a static flowchart — it does not adapt to a new patient. Also, the doctor's questions are chosen based on medical knowledge; the tree's questions are chosen purely by statistics — which split best separates the classes. Finally, a doctor can explain her reasoning in terms of medical mechanism; a tree can only say 'this split had the most statistical separation.'

A decision tree is a model that recursively splits the feature space on single-feature thresholds, producing a set of if-then rules that are readable end to end, and that classify or predict by following the path from root to leaf.

You have not learned any machine learning concepts yet, so this is your first building block. But you will soon learn about the bias-variance tradeoff, which explains why an unpruned tree overfits — it has too much capacity and memorizes noise. You will also learn about K-Nearest Neighbours, which is a non-parametric method that, unlike a tree, has no training phase and stores all data. Trees are the foundation for Random Forests, which correct a tree's instability by averaging many trees. The neighbouring concept of Gini and Information Gain describes the exact criteria a tree uses to choose splits, which is the heart of this page. Later, you will see that trees are a classical model — often the right choice for small tabular data, before reaching for deep learning.