CNN Visualization for Interpretation

You are about to learn how to look inside a convolutional network and see what it has actually learned. This skill is what turns a network from a black box into something you can interrogate. It unlocks the next stage of your study: knowing why a model made a decision is essential for debugging it, trusting it in high-stakes settings like medical imaging or fraud detection, and explaining it to a non-technical stakeholder. After this topic, you will be able to identify which parts of an input image most influenced a prediction, something that is directly useful in computer vision and multimodal AI.

A convolutional network learns by looking at images through a series of filters. Each filter is a small pattern, like a tiny stamp, that slides across the image and highlights where that pattern appears. In the early layers of the network, these filters learn simple patterns: edges, colours, or blobs of light and dark. In later layers, the network combines those simple patterns into more complex ones, like a corner, a wheel, or eventually an entire object. When you want to interpret what the network is doing, you have three main tools. First, you can look at the filters themselves: each one is just a small table of numbers, and you can display those numbers as an image to see what pattern the filter is looking for. Second, you can look at the activation maps: after a filter slides over the image, the network produces a new image (smaller or same size) showing how strongly the filter matched at each location. Third, you can look at what region of the original image made the network decide 'dog' vs 'cat' by using techniques like class activation maps, which highlight the pixels that were most important. These three views give you a window into the network's thinking at different levels of abstraction.

Imagine you are a detective investigating how a team of art critics judges paintings. Each critic has a specialty: one only looks for straight lines, another for the colour red, another for curves. When a critic sees a painting, they walk across it, looking for their specialty. After they finish, they give you a map of the painting with marks where they found their pattern. You, the detective, want to know why a particular painting was judged 'modern art'. So you ask each critic for their map. The lines-critic shows you a map full of marks on the building edges, the red-critic shows marks on the red circle, the curves-critic shows marks on the spiral. You can also ask directly what each critic's specialty is, by asking them to draw a generic example of what they look for. And finally, you can ask, 'Which part of this painting most convinced you it was modern?' They would point to the spiral, saying that is the defining feature. Now, in a convolutional network, the critics are filters, their maps are activation maps, and the pointing finger is the class activation map. This analogy works well, but it breaks down in one way: a human critic can tell you their specialty in words ('I look for curves'), but a neural network filter is just numbers—it cannot explain itself. The numbers are all we have, and we must translate them into something visual to understand them.

CNN visualization for interpretation is the process of inspecting the learned filters, activation maps, and input regions that most strongly drive a convolutional network's prediction, in order to understand what features the network has learned and why it made a particular decision.

You are studying this as part of Convolutional Networks. Your library notes mention that this is one of the more successful interpretability approaches, and it connects directly to your notes on Convolutional Layers (filters live there), Convolution Arithmetic (filter sizes affect what you see), Receptive Fields (later layers see more context, which explains why their filters are more complex), and Pooling as a Statistic (pooling affects the resolution of activation maps). It also connects to your notes on Fourier Analysis, because networks are known to learn low-frequency patterns first, and you can see that tendency in how filters evolve during training.