Convolutional Layers

Convolutional layers are the workhorse of modern vision and increasingly of one-dimensional signal processing. They let a model learn one small pattern (an edge, a curve, a texture) and then detect that same pattern anywhere in an image — something a fully connected layer cannot do efficiently. Mastering this unlocks: Convolution Arithmetic (how output sizes are computed), Pooling as a Statistic (how to summarise regions), Receptive Fields (how later layers see more context), and CNN Visualization for Interpretation (inspecting what the learned filters actually detect). The same idea — a learned filter sliding across a signal — also underlies speech recognition, sensor readings, and time-series forecasting. Understanding this node is the bridge from the single neuron you already know to the entire world of vision models.

Imagine you are looking at a photograph of a cat. You recognise the cat not by checking every single pixel against every other pixel, but by spotting small, repeated patterns: the curve of an ear, the texture of fur, the shape of an eye. Wherever you look in the photo, you use the same mental templates. You don't need one template for 'ear in the top-left' and another for 'ear in the bottom-right' — the same template works because an ear is an ear, regardless of position. A convolutional layer does exactly this. It takes a small grid of numbers (the filter, or kernel), slides it across the entire image, and at each position computes how well the image patch matches the filter. A high match means the pattern is there. The critical trick: the filter's values are not hand-designed — they are learned from data. So the model figures out which patterns matter (edges in early layers, then curves, then composites) and then applies each learned pattern uniformly across the whole image. It is like having a stamp that imprints the same shape anywhere on a map — once you have the stamp, you can press it everywhere. The same filter weights are reused at every position, which is the weight sharing that gives translation equivariance and massive parameter savings.

Think of a detective scanning a crowd photo looking for a specific person. He doesn't memorise the whole photo pixel by pixel. Instead, he has a small portrait — a template — of the person's face. He systematically slides that template over the photo, stopping at every position to compare. Does this patch match the template well? A little? Not at all? He records a score for each position, building a map of 'where does the person appear' — a heat-map of matches. Now, crucially, the detective uses the same template everywhere. He doesn't need one template for faces in the left half and another for the right half. This is the power of weight sharing: one learned pattern works everywhere. The analogy breaks down when you remember that the detective's template is fixed. In a convolutional network, the template (the filter's numbers) is learned from data. We start with a random blurry template and adjust its numbers over and over until it produces high scores precisely where the pattern actually exists and low scores elsewhere. The detective would do well to update his template as he learns the person's face from different angles. A second breakdown: the detective has only one template, but a convolutional layer typically has many filters — potentially hundreds — each learning a different pattern, so it can detect edges, circles, textures, and more, all in parallel.

A convolutional layer applies a set of small, learned grids of numbers (filters) across an input signal, one position at a time, computing at each position the degree of match (the dot product between the filter and the local patch), and producing an output map of these matches for each filter.

You have just learned how a single artificial neuron works: it takes a list of inputs, multiplies each by a weight, adds a bias, and squashes the result. A convolutional layer is a special way of arranging many neurons. Instead of every neuron connected to every input (as in a dense layer), each neuron in a convolutional layer is connected only to a small, local patch of the input — and crucially, the same set of weights is reused across all positions. This is the 'shared weights' idea from your notes on weight sharing and translation equivariance. It is the conceptual bridge from the fully connected neuron to the world of vision.