Linear Regression

You use linear regression when you need to predict a number based on other numbers, such as estimating next month’s sales from advertising spend or guessing a house’s price from its square footage. It is the simplest way to see how changes in your inputs lead to changes in your output. For example, if you have data showing that each additional bedroom adds roughly $20,000 to a home’s value, linear regression gives you that clear, direct relationship. Mastering it allows you to interpret exactly which factors matter and by how much, rather than treating the prediction as a black box.

Imagine you want to predict the price of a house. You look at three numbers for each past sale: the size in square feet, the number of bedrooms, and the age of the home. Linear regression asks: 'What is the steady, straight-line pattern that connects these inputs to the final price?'

For just one input, like size, you draw a straight line through the dots on a graph where the x-axis is size and the y-axis is price. This line might show that for every extra 100 square feet, the price goes up by $15,000.

When you have multiple inputs (size, bedrooms, age), the 'line' becomes a flat sheet or surface stretching through the data in many dimensions. We call this shape a hyperplane. It is simply a flat surface that exists even when you can’t easily visualize more than three dimensions. The goal is to find that specific flat surface that comes as close as possible to all the past house prices.

How do we define 'as close as possible'? We look at the vertical distance between each actual sold price and where our flat surface predicted it would be. This difference is the error. To avoid positive errors canceling out negative ones, and to punish large mistakes more severely, we square each of these distances (multiply the error by itself). Then, we add up all those squared errors for every house in your data. The 'best' hyperplane is the one that makes this total sum of squared errors as small as possible.

Think of a waiter balancing a long tray with multiple plates on it. Each plate represents an input factor: one plate holds the weight of 'size', another holds 'bedrooms', and another holds 'age'. The waiter’s arms act as the pivot points, adjusting how much each plate contributes to the final tilt of the tray. The total tilt of the tray corresponds to the predicted price.

To train the waiter, you give them hundreds of examples of actual houses and their real prices. They adjust the position of the plates (the coefficients) until, across all those examples, the tray’s tilt matches the actual weight of the house as closely as possible.

The analogy holds because both involve balancing contributions to reach a total. It breaks down only in that a waiter physically feels gravity and balance, whereas linear regression mathematically calculates positions to minimize distance, allowing for negative contributions (like older houses lowering the price) which are harder to visualize on a physical tray.

Linear regression is a method of finding the flat surface that best fits historical data by adjusting its angles so that the total squared difference between its predictions and the actual values is as small as possible.

This concept serves as the foundational grammar for many other prediction models. Its neighboring topics include Logistic Regression, which uses the same flat-surface logic but squashes the result to predict a yes-or-no outcome instead of a number, and Polynomial Regression, which allows the surface to curve rather than stay flat to capture more complex relationships.