Forward Propagation

Forward propagation is the first half of training a neural network — it’s how the network turns inputs into predictions. You already know about the artificial neuron: inputs multiplied by weights, added to a bias, then squashed by an activation function. Forward propagation is simply doing that to every neuron in every layer, moving from left to right. Once you master this, you can build the three-layer network from scratch, understand how a shape mismatch crashes your code, and then learn backpropagation — the backward pass that adjusts the weights using the intermediate activations you saved. Forward propagation is the engine that runs every time you use a trained model to make a prediction, from a loan approval to a ChatGPT response.

Imagine a production line in a factory. Raw materials come in at one end; finished goods come out at the other. Each work station takes what it receives, does its own transformation, and passes the result to the next station. In a neural network, the raw materials are your input numbers (like a loan applicant's income, debt, and years at the job). Each layer is a work station. It takes the numbers from the previous layer, multiplies them by its own weights, adds its bias, and applies an activation function to produce a new set of numbers. Those numbers become the input to the next layer. This chain continues until the last layer produces the final output — the network's prediction. The key is that the transformation at each layer is the same small operation you already know: multiply, add, squish. Forward propagation is just doing that operation over and over, layer after layer, from the input to the output. Nothing more, nothing less.

Think of a relay race where each runner doesn't carry the baton but transforms it. At the start, you have a number (say, 5). The first runner has a set of rules: 'take the number, multiply by 0.9, add 0.2, then if the result is positive, keep it as is, otherwise change it to zero.' The runner hands the resulting number to the next runner, who applies a different set of rules. And so on until the final runner produces the race result (the prediction). Each runner is a neuron in a layer; the rules are the weights, bias, and activation function. In this relay, the baton changes value at every hand-off, but the baton itself is just a number (or a list of numbers when you have multiple neurons in a layer). The whole relay is forward propagation. Why does this work? Because each runner can emphasize different aspects of the signal. The first runner might focus on the magnitude, the second on the sign, the third on whether it exceeds a threshold. By the end, the number has been reshaped so that it encodes meaningful information for the final decision. But here's where the analogy stops: in a relay, each runner runs the same path; in a network, each neuron has its own unique weights, and the 'rules' are learned from data, not set by a coach. Also, the final output is usually a number between 0 and 1 (with a sigmoid) or a set of scores (with softmax), not a race time. The baton (the data) is transformed, not just carried.

Forward propagation is the process of computing a neural network's output by passing the input through each layer sequentially, where each layer performs a linear transformation (matrix multiplication with weights, plus a bias) followed by an activation function, and the output of one layer becomes the input to the next.

This builds directly on the Artificial Neuron concept. You know that a single neuron computes a weighted sum of its inputs, adds a bias, and applies an activation. Forward propagation is just that done for many neurons, organized into layers. You also know about activation functions (like sigmoid, ReLU) from the Activation Functions topic. The mathematics here is linear algebra (matrix multiplication) and algebra (the weighted sum). You'll need to understand how matrices multiply, which we'll teach from scratch. The output of forward propagation is later used by backpropagation to compute gradients — so you'll need to remember the intermediate activations (we'll discuss that in the code section). After this, you'll be ready to study how networks learn (Gradient Descent, Backpropagation) and then move on to Convolutional Networks, Attention, and Transformers.