Computational Graphs

This explains how a computer figures out which part of a complex calculation caused an error, so it can fix it. Imagine you are baking a cake, and it turns out too dry. You need to know if the oven was too hot or if you added too little milk. A computational graph is a map of every step in your process — from mixing flour to pouring batter to checking the temperature. By looking at this map, the computer can trace the 'dryness' back through each step to find exactly which ingredient or setting changed how much it matters. Without this map, the computer would have to guess and re-bake the cake thousands of times just to find a small fix. With it, you get an exact answer in one go. This is why your software can learn from its mistakes so quickly: it isn't guessing; it is reading the recipe backward.

Let us look at a specific example. Suppose you are calculating the total cost of buying apples and oranges. The steps are simple:
1. You buy 3 apples, and each apple costs $2. (Step 1: Multiply quantity by price for apples). Result: $6.
2. You buy 4 oranges, and each orange costs $1.50. (Step 2: Multiply quantity by price for oranges). Result: $6.
3. You add the cost of apples to the cost of oranges. (Step 3: Add the two results together). Final Total: $12.

Now, imagine the final total is wrong. You want to know how much changing the price of an apple would change that final $12 total.

In a computational graph, this looks like three boxes connected by arrows. Box 1 outputs $6 into Box 3. Box 2 outputs $6 into Box 3. Box 3 outputs $12.

To find the 'importance' of the apple price to the final total, you start at the end ($12) and walk backward:
- At Box 3 (the addition), you ask: 'If I change the money coming from Box 1 by a tiny bit, how much does my total change?' Since you are just adding them together, if the apple cost goes up by $0.01, the total goes up by $0.01. So the link from Box 1 to Box 3 has a 'strength' of 1.
- Then you look at Box 1 (the multiplication). You ask: 'If I change the price of one apple by $0.01, how much does Box 1’s output ($6) change?' Since you bought 3 apples, changing the single-unit price by $0.01 changes the total apple cost by $3 * $0.01 = $0.03. So the link from the apple-price input to Box 1 has a 'strength' of 3.
- To find the total effect, you multiply the strengths along the path: 1 * 3 = 3. This means for every $0.01 increase in the apple price, the final total increases by $0.03.

You did this by walking backward through the steps you already took, using the numbers from those steps to figure out how sensitive each step was to its own input. You never had to re-do the whole calculation; you just traced the influence.

Think of a computational graph as a system of pipes carrying water. The forward pass is turning on the tap: water flows from the source, through various valves (the steps), and fills a tank at the end (the final answer). Each valve controls how much water passes through based on its setting.

The backward pass is like checking which valves are most responsible for the current water level in the tank. You start at the tank and look upstream. If you tighten one valve slightly, how much less water reaches the tank? You check each pipe connected to the tank, then follow those pipes back to where they came from. Because you know exactly how much water is currently flowing through each pipe (from the forward pass), you can easily calculate how a tiny adjustment at any point will ripple backward to affect the final level. The graph is the map of these pipes, ensuring you don't miss any connections or double-count flows where pipes split and merge.
Caveat: unlike real water, changing a parameter in one part of the calculation does not physically change the previous numbers; it only changes your understanding of how much they *would* matter.

A computational graph is a diagram that breaks a calculation into individual steps, linking each step to the next with arrows. It allows you to work backward from the final result to determine exactly how sensitive that result is to every single input used in the process.

This concept sits beside 'Derivatives', which is just a formal way of describing how sensitive one number is to changes in another — essentially, the 'strength' or 'slope' of influence between two connected steps. It also connects directly to 'Automatic Differentiation', which is the method computers use to automatically trace these backward paths through any complex calculation without you having to do the algebra yourself.