Rule-Based Classifiers

You are starting your journey into machine learning from scratch. Rule-based classifiers are the first and simplest kind of classifier you will meet, and they are the perfect entry point because they require no mathematics beyond arithmetic and no programming beyond basic Python. They are also the foundation for more advanced ideas: every decision tree is a set of rules, every random forest is a collection of trees, and even the most complex neural network can be inspected as a set of rules after training. Learning rule-based classifiers will give you the vocabulary and intuition you need to understand how all classifiers work: they all take input values, apply some computation, and produce a label. By the end of this page, you will be able to hand-build a simple classifier for a real problem, explain what 'training' means in this context, and recognise where rules are genuinely useful versus where they fail.

Imagine you are a doctor in a small clinic. A patient comes in with a fever, a cough, and a sore throat. You don't run any tests—you just look at the symptoms and decide: 'This is a cold' or 'This is the flu.' How do you do that? You have a set of rules in your head: if the fever is high and the cough is dry, it's probably the flu; if the fever is low and the cough is wet, it's probably a cold. You don't calculate anything mathematical; you just apply rules. That is exactly what a rule-based classifier does. It looks at the input values (the symptoms) and applies a series of if-then rules to decide which class (the disease) the input belongs to. The rules are explicit, which means a human can read them and understand exactly why the machine made its decision.

Now, why do we call it a 'classifier'? Because its job is to classify—to put things into categories. Think of sorting mail: letters go in one pile, packages in another. You classify each item based on rules: if it's flat and rectangular, it's a letter; if it's bulky, it's a package. A rule-based classifier does the same thing with data. It might classify emails as spam or not-spam, or customers as likely to churn or not.

The key idea is that the rules are explicit. They are written down in plain language, like 'IF the patient has a high fever AND a dry cough, THEN diagnose the flu.' This is different from many other machine learning methods, where the decision process is hidden inside complex numbers and equations. With rules, you can open the box and see exactly what's inside. That transparency is valuable in situations where you need to justify a decision, like a bank denying a loan or a hospital recommending a treatment.

Think of a passport control officer at an airport. The officer has a set of rules written in a manual: if the passport is valid, if the visa is correct, if the person's name matches, then allow entry; otherwise, deny entry. The officer does not compute a score or use any statistical model—they just follow the rules step by step. This is rule-based classification: a sequence of if-then checks that lead to a decision.

The beauty of this analogy is that it shows how rules can be hierarchical. The officer might first check if the passport is from a visa-free country. If yes, they skip the visa check. If no, they check the visa. This is a tree structure, which is exactly how many rule-based classifiers are organised. Each rule is a branch, and the final decision is a leaf.

Now, where does this analogy break down? A passport officer only has a handful of rules, and they can handle exceptions easily. In machine learning, rule-based classifiers can have thousands of rules, and they are often generated automatically from data, not written by hand. Also, a human officer can use common sense and context—if a rule leads to an absurd outcome, they can override it. A rule-based classifier cannot; it will follow its rules blindly, even if they are wrong. So while the analogy captures the essence of rules, it does not capture the scale, the automatic generation, or the inflexibility of a real system.

A rule-based classifier is a model that assigns a class label to an input by applying a set of explicit if-then rules, where each rule matches the input's feature values to a condition and maps it to a decision.

You are starting from scratch, so you haven't mastered any other machine learning concepts yet. But this page is your first step. Rule-based classifiers are the simplest form of a classifier, and they will connect to everything you learn later. For example, decision trees (which you'll see in the 'Decision Trees' topic) are essentially rule-based classifiers with a specific structure. The rules you learn here will reappear there. Also, the idea of 'features' (the input values) and 'labels' (the output classes) is used in every classifier, from K-Nearest Neighbours to neural networks. So by mastering rule-based classifiers, you are building the foundation for all of machine learning.