Two-Class and Multiclass Classifiers

You need to be able to sort things into groups based on what you can see or measure. Sorting into two groups, like 'spam' or 'not spam', is the basic skill. Sorting into three or more groups, like identifying a disease from a leaf image, uses that same basic skill but with more options. Understanding how these sorting tools work helps you choose the right one for your job, so you can score their accuracy correctly and predict outcomes like whether a customer will leave.

Imagine a piece of paper with dots on it. Each dot has a label, like 'A' or 'B'. Two-class classification is about drawing a line that puts all the 'A' dots on one side and all the 'B' dots on the other. A classifier is simply that line plus a rule: if a new dot lands on the 'A' side, call it 'A'; if it lands on the 'B' side, call it 'B'. Multiclass classification happens when there are three or more kinds of dots. One line cannot separate three groups clearly. You can use different strategies. One strategy is to draw several lines, each one separating one specific kind from all the others combined. Another strategy is to draw a line between every possible pair of kinds and let them vote on the result. A third strategy is to use a model that handles all the kinds at once without needing separate pairwise comparisons.

You are sorting fruit at a market. When you only have apples and oranges, you look at features like color and shape and draw a mental line: 'if it is red and round, it is an apple; if it is orange and oval, it is an orange.' This is two-class classification. Now imagine you must also sort bananas, mangoes, and grapes. You cannot use a single line for all of them. You could use the 'one-vs-rest' strategy: ask first, 'is this an apple?' If yes, set it aside. If no, ask, 'is this an orange?' Set that aside. Continue until you find what fits or declare it 'miscellaneous.' This is simple but can get confused if two fruits look similar, like a green apple and a lime. Alternatively, you could use the 'one-vs-one' strategy: compare the fruit to an apple, then to an orange, then to a banana, voting for each winner until one kind has the most votes. This is more careful but slower. Or you could develop a single mental model that looks at all features at once and directly names the fruit, which is often the best approach if you can build it.

A classifier is a model that assigns an input to one of several categories. Two-class classification distinguishes between exactly two options, while multiclass classification distinguishes among three or more options using methods like comparing each class against all others, comparing every pair of classes, or using a single model designed for multiple groups.

This connects to Logistic Regression, which calculates the probability of one of two outcomes. It also relates to Decision Trees and K-Nearest Neighbours, which are tools that naturally handle multiple categories by splitting data into groups based on feature values. The concept of separating classes is similar to the dividing line between categories in Rule-Based Classifiers, but here the line is automatically learned from the data rather than manually defined.