Multi-Layer Perceptrons

This is the baseline architecture of deep learning. Once you understand a multi-layer perceptron, you understand the skeleton of every modern neural network — including the Transformer behind ChatGPT. This concept unlocks the rest of deep learning: convolutional networks for images, transformers for text and speech, and the 'deep' in deep learning is just adding more layers to what you'll build here. You'll also see why MLPs are still the go-to choice for tabular data — spreadsheets, sensor logs, financial records — where there is no natural grid or sequence to exploit.

A multi-layer perceptron (MLP) is a stack of neurons. You already know a single neuron: it takes several inputs, multiplies each by a weight, adds a bias, then squashes the result with an activation function. An MLP arranges neurons in layers: the input layer is just your data (each input value becomes one neuron, but these don't compute anything — they just hold the values). The output of each neuron in the first hidden layer becomes an input to every neuron in the next layer. So each neuron in a hidden layer takes inputs from all the neurons in the previous layer, multiplies each by its own weight (its own importance), adds its own bias, and activates. This repeats layer by layer, until the final output layer produces the answer. The key idea is that the hidden layers gradually transform your raw input into increasingly abstract representations — first simple combinations (e.g., 'income is high and debt is low'), then more complex patterns (e.g., 'this loan applicant looks like a good risk because income is high, debt is low, and tenure is long, but there's a red flag about recent defaults'). The magic is that the weights are learned automatically from data — no one manually designs these patterns.

Think of a manufacturing assembly line. Raw materials enter at one end, and a finished product exits at the other. Each workstation takes the output of the previous station, does a specific operation on it, and passes the result to the next. An MLP is that assembly line, but the operations are all the same kind: each neuron is a worker. The worker receives a set of inputs (the outputs of the previous workers), weighs them according to how important they are (weights), has a personal bias (how easy it is to get them to say 'yes'), and then applies their own nonlinear 'squash' (activation) that decides how strongly they pass their enthusiasm forward. Critically, no one designs these workers — they are all trained. At first, the workers compute random nonsense; the assembly line produces garbage. But during training, the weights are adjusted step by step, so the line gradually produces the desired output. Where does the analogy break? In a real assembly line, each station does a fixed, bespoke operation. In an MLP, every neuron does the same simple operation (weighted sum + activation). The power comes from the sheer number of these simple units and their connections — the line has, say, hundreds of workers at each stage, and each worker talks to all workers in the next stage. Also, in a real factory, the flow goes only one way — and that's true for an MLP too: it's feedforward, data flows from input to output, no loops.

A multi-layer perceptron is a feedforward neural network with one or more hidden layers between the input and output, where every neuron in each layer is connected to every neuron in the next layer, each connection carrying a weight, and each neuron applying a nonlinear activation function to its weighted sum plus bias.

This builds directly on the artificial neuron: each neuron here is exactly that unit. You also need to be comfortable with the dot product (multiplying pairs of numbers and summing) because each neuron's computation is just a dot product between its inputs and its weights, plus a bias. The activation functions you've studied (sigmoid, ReLU) are what make depth meaningful — without them, the whole MLP collapses into a single linear transformation (a giant matrix multiplication) and you've lost all the power of stacking layers. This connects forward to backpropagation: training an MLP is what backpropagation was invented for. And it's the foundation of every architecture you'll see next — convolutional networks replace some connections with local ones, transformers replace layers with attention, but the fundamental idea of composing transformations through layers is the MLP's.