Linear Transformations

Linear transformations are the single most important idea in linear algebra, and linear algebra is the language of machine learning. Every neural network layer is a linear transformation followed by a nonlinearity. Understanding transformations now means that later, when you meet matrix multiplication, you will see it as composing maps rather than as a row-by-column ritual. This unlocks: Matrix Multiplication (the arithmetic behind everything), Determinants and Inverses (how much a transformation stretches space, and whether it can be undone), Eigenvalues and Eigenvectors (the directions a transformation merely stretches), and Singular Value Decomposition (factoring any transformation into a rotation, a scaling, and another rotation — the basis of PCA and model compression).

A linear transformation is a rule that takes every point in space and moves it to another point, but with strict restrictions: straight lines must stay straight, parallel lines must stay parallel, and the origin must stay fixed. Think of stretching a rubber sheet that is pinned at the centre: you can pull it, squash it, rotate it, flip it over — but you cannot tear it, and you cannot bend it into a curve. Any such rule is completely described by what it does to two simple starting arrows: the one pointing one unit right, and the one pointing one unit up. Because every other point is just a combination of those two arrows, the transformation's effect on the two arrows determines its effect on everything. This is why a 2x2 table of numbers (a matrix) can represent the whole transformation.

Imagine a graph-paper grid drawn on a rubber sheet, pinned at the bottom-left corner (the origin). A linear transformation is any way you can stretch, squash, rotate, flip or shear that sheet — but you cannot fold it, tear it, or move the pin. After the transformation, the grid lines are still straight, parallel lines are still parallel, and the origin is still where it was. The entire rule for moving every point is captured by where two particular points land: the point (1,0) — one square to the right — and the point (0,1) — one square up. Suppose (1,0) lands at (2,1) and (0,1) lands at (−1,3). Then the point (2,3), which is 2 copies of (1,0) plus 3 copies of (0,1), must land at 2 copies of (2,1) plus 3 copies of (−1,3) = (4,2) + (−3,9) = (1,11). This is exactly how matrices work. Where the analogy breaks down: not every grid you can draw with straight lines is a linear transformation. The origin must stay fixed — if the whole sheet slides, even without bending, that is an affine transformation, which is a linear transformation plus a shift. That extra shift is what the bias term in a neural network layer provides. Also, real-world rubber can tear, but a linear transformation never tears: it is always completely smooth and defined for every point.

A linear transformation is a rule that takes a vector as input and produces a vector as output, such that adding two inputs first and then transforming gives the same result as transforming each input separately and then adding, and scaling an input first and then transforming gives the same result as transforming and then scaling.

You have not met vectors formally yet, so think of a vector as an arrow from the origin to a point, or simply as a list of coordinates like (2,3). This concept sits at the heart of linear algebra, the branch of mathematics that studies vectors and the transformations between them. Once you have this, the next step is matrix multiplication, which is just a compact way to write what a linear transformation does to every point. Later, determinants will measure how much the transformation stretches or squashes area (volume), and eigenvectors will find the directions that the transformation only stretches. In machine learning, every layer of a neural network is a linear transformation followed by a nonlinear step, so this concept is the first step to understanding how data flows through a model.