Decision Jungles

Understanding decision jungles opens the door to a whole layer of machine learning engineering: making models small enough to run on a phone, a smartwatch, or a sensor without a cloud connection. This is the difference between a demo that needs a server and a product that works in your pocket. You will also deepen your understanding of how decision trees work and why they are so widely used. This concept is a bridge to later topics like model compression and on-device AI, which you will meet in the advanced parts of this system.

Think of the difference between a tree and a jungle. In a regular decision tree, every path is unique: starting at the trunk, each branch splits, and each leaf can only be reached by its own distinct route. If two different routes end up at the same conclusion, they still exist as separate branches, even if they are identical. A decision jungle is a forest where trees are allowed to merge. Instead of having two separate branches that both lead to the same place, they share a common path. This sounds like a small change, but it has a big effect on memory. In a jungle, a node can be reached from more than one starting point, so you do not need to store the same part of the tree twice. The result is a model that captures the same patterns with fewer nodes, which means it takes up less space in memory and can run faster on a device with limited resources. The trade-off is that building a jungle is more complex than building a regular tree, because you have to decide where to merge paths without losing accuracy. But when space is tight, a jungle can be the difference between a model that fits and one that does not.

Imagine a large city with a network of one-way streets. A regular decision tree is like a city with no shortcuts: every trip from the suburbs to the centre must follow a unique, fixed route, and no two trips ever share a road. If two different neighbourhoods both need to get to the same central square, they have to build two entirely separate roads that never meet, even though they end at the same place. That is a waste of space and materials. A decision jungle is like a city with a well-designed road network: roads can merge, so two different starting points can share a highway for the last stretch into the centre. This saves construction materials and makes the journey faster because there are fewer roads to maintain. But there is a catch: designing a city with merging roads is much harder than designing a simple tree. You have to plan where to merge so that the traffic still flows efficiently. If you merge too early, you might cause a bottleneck, and if you merge too late, you have wasted the opportunity. In a decision jungle, similar reasoning applies: the structure of the graph is learned, and the merge points are chosen to minimise memory while preserving accuracy. The analogy breaks down if you think of the roads as being able to split and merge freely without cost — in a jungle, the graph is directed acyclic, meaning that after a merge, the path cannot go backwards, and cycles are not allowed. So the city is a one-way street system with no loops, which is exactly what makes it a DAG.

A decision jungle is a collection of decision trees where the trees are replaced by directed acyclic graphs (DAGs), allowing different paths to merge, which reduces the total number of nodes needed to represent the same decision function.

This concept builds directly on your notes on Decision Trees and Gini and Information Gain. A decision tree is a simple, interpretable model that recursively splits data based on single-feature thresholds. A forest is an ensemble of such trees. A jungle takes that one step further by allowing the trees to share nodes, which is a form of model compression. This is related to ensemble methods and the bias-variance tradeoff from your Machine Learning Foundations notes: by merging, you trade a small amount of model capacity for a large reduction in memory, which is often a good trade when the alternative is not being able to deploy the model at all. The idea of merging also appears in other areas of machine learning, such as pruning and weight sharing, but jungles are unique in allowing structure to be shared at the graph level.