Feature Stores

A feature store lets you build the same input representation for training and for live predictions, which stops the model from silently degrading in production. It is the only component in the AI stack that survives a change in data modality (text to images), a model upgrade (from a simple linear model to a deep network), and a shift in time horizon (from daily batch to real-time). Once you master this, you can understand how MLOps teams bridge experimentation and deployment, and you will be ready for the next step: Data Contracts and the Semantic Layer, which formalise the agreement between data producers and models.

When you train a model, you feed it rows of numbers, each column a feature. When you use that model on a live user, you must feed it exactly the same kind of numbers, computed in exactly the same way. The naive way is to write separate code for training and for serving. But the two often drift apart: a data scientist writes a transformation to compute 'average monthly spending' for training, and a site reliability engineer rewrites the same logic a different way for production. The model then sees a slightly different input than it was trained on, and its predictions get worse without anyone noticing. A feature store fixes this by computing the features once, in one place, and serving them to both training and inference. It also stores the features so they are computed once, not every time, and it ensures each training row uses the value that was known at the time of the prediction, not a future value that would leak information. The store is shared infrastructure: a single system that every team uses, so the definition of 'average monthly spending' is identical everywhere.

Think of a shipping warehouse between two factories. The training factory and the serving factory each need the same components (features), but if they order those components from different suppliers (codebases), they will get different parts — one is metric, one is imperial, so they cannot fit together. A feature store is the central warehouse that both factories order from. It stores the parts in one standard form, prepared in one way, and ships the same part to both. The warehouse also keeps a log of when each part was in stock, so if a product needs 'inventory as of last Tuesday', it gets exactly that, not today's stock level. This log is the point-in-time correctness — it prevents the assembly line from accidentally using a future part to build a past product. The analogy breaks down where the feature store must also compute: it does not just store pre-made parts, it also has a machine that builds each part on demand, and it can run that machine ahead of time for training or right on the spot for a live prediction. The warehouse also holds many parts that different factories use, so it becomes the single source of truth for what a 'part' even is.

A feature store is shared infrastructure that computes, stores, and serves feature values consistently for both model training and live inference, while ensuring each value is correct for the time it is used.

This sits inside Feature Engineering, which is the work of turning raw data into numbers a model can use. You already know that leakage — using information unavailable at prediction time — is the most damaging error in machine learning; the feature store is the main mechanism to prevent it. It also connects to Data Quality and Data Contracts, because the store enforces the schema of the features, and to MLOps, because deploying a model is mostly deploying its feature pipeline. The neighbouring notes on Feature Selection and Class Imbalance assume you have clean features; the store is what makes them clean consistently.