In words
What it is, why it matters, and what it is like.
Why am I learning this?
Understanding linear transformations lets you see matrix multiplication not as a tedious row-by-column ritual, but as the composition of simple shape shifts. This perspective is essential because every layer in a neural network begins with a linear transformation followed by a nonlinearity—a rule that bends space to make patterns visible. By mastering this idea now, you gain the foundation for grasping how machines process data. Specifically, it clarifies what determinants and inverses measure: how much a transformation stretches or shrinks area and whether that stretching can be perfectly reversed. It also reveals why eigenvalues and eigenvectors matter: they identify the specific directions in space that a transformation simply stretches without rotating, which is key to reducing complex data into its most important components. Finally, it explains the role of singular value decomposition, which breaks any complex shape shift into a rotation, a scaling, and another rotation—the mathematical basis for compressing information and filtering noise.
The idea, in plain terms
A linear transformation is a rule that moves every point in space to a new location, but it must follow three strict rules: straight lines must remain straight, parallel lines must remain parallel, and the starting point (the origin) must stay exactly where it is. Think of a grid drawn on a perfectly flexible rubber sheet that is pinned at its center. You can stretch, squash, rotate, or flip the sheet, but you cannot tear it, fold it, or bend it into curves. Because the origin is pinned, any such movement is entirely determined by what happens to just two simple arrows: the arrow pointing one unit to the right and the arrow pointing one unit up. Since every other point in space is just a combination of these two arrows, knowing where they land tells you exactly where every other point goes. This is why a 2x2 table of numbers (a matrix) can completely describe the transformation: it simply records the new locations of those two fundamental arrows.
An analogy
Imagine a graph-paper grid drawn on a rubber sheet, pinned firmly at the bottom-left corner (the origin). A linear transformation is any way you can stretch, squash, rotate, flip, or shear that sheet without tearing it, folding it, or moving the pin. After the movement, the grid lines are still straight, parallel lines remain parallel, and the origin is still fixed. The entire rule for moving every point is captured by where two particular points land: the point (1,0)—one square to the right—and the point (0,1)—one square up. Suppose (1,0) lands at (2,1) and (0,1) lands at (-1,3). Then the point (2,3), which is 2 copies of the right-arrow plus 3 copies of the up-arrow, must land at 2 copies of (2,1) plus 3 copies of (-1,3). The calculation follows two steps: first, multiply the components: 2 times (2,1) gives (4,2), and 3 times (-1,3) gives (-3,9). Second, add the results: (4,2) + (-3,9) equals (1,11). This arithmetic is exactly how matrices work. The analogy holds perfectly for this type of rigid, origin-fixed stretching. However, note that in neural networks, we often allow the sheet to slide slightly before bending it; that initial slide does not change the shape but shifts the position, a step handled separately by a bias term.
Definition
A linear transformation is a rule that maps every input vector to an output vector such that the transform of a sum equals the sum of the transforms, and the transform of a scaled input equals the scale times the transform of the input. Formally, it is a function T where T(u + v) = T(u) + T(v) and T(cu) = cT(u) for all vectors u, v and scalars c.
Where this sits
You can think of a vector as an arrow from the origin to a specific point, or simply as a list of coordinates like (2,3). Linear transformations are the core operation in linear algebra, the branch of mathematics that studies these arrows and how they move. The next logical step is matrix multiplication, which is merely a compact notation for applying a linear transformation to every point simultaneously. This leads directly to determinants, which calculate how much area or volume the transformation stretches or shrinks, and eigenvectors, which identify the specific directions that are only stretched, never rotated.
