Model Serving APIs

This concept unlocks the rest of MLOps. Without a serving API, your model is a file on a disk — nobody can use it, and you cannot monitor it. Once you understand how a model is exposed as an API, you can build on it: releasing a new version safely (Model Registry and Versioning), catching training-serving skew, setting up drift detection, and adding human-in-the-loop feedback. Every real AI system you will work with — from a chatbot to a fraud detector — talks to its model through an API like the one you will build here.

Think of a model as a kitchen. You have a recipe (the model), ingredients (the input features), and a chef (the trained model itself). Without a waiter, the chef never receives orders and the kitchen never sends out food. The API is the waiter: it takes the order, checks that it is complete and makes sense, passes it to the chef, and brings back the dish. But the API is more than just a messenger. It also decides who is allowed to order, what happens if the kitchen is closed, and how to tell the rest of the restaurant that the kitchen is ready for new orders. In the same way, a model serving API is the agreed-upon door through which all requests enter and all responses exit. It is the model's real interface with the world, and it is the first thing that catches mistakes before they waste compute or cause a wrong prediction.

Imagine a restaurant called 'Model House'. The kitchen inside has a secret recipe for predicting loan defaults. Customers (client apps) don't walk into the kitchen — they order from a menu at the counter. The counter is the API. It has a menu that says: 'Give me three numbers: income, debt, and years at job. I'll give you a score between 0 and 1.' If a customer hands the counter a piece of paper with 'high income' written on it, the counter says, 'Sorry, I need numbers, not words' — that's schema validation. If the kitchen is closed for the day, the counter says, 'We're closed today' — that's a health check. And if the restaurant has changed its recipe, the counter might say, 'Oh, you're using the old menu? Here's the old version, with the old recipe' — that's versioned endpoints. The counter also tells the manager whether the kitchen is ready to serve (readiness) and whether it's still alive (liveness). Now, where does the analogy break down? A restaurant counter is human and can improvise. An API is rigid — it follows exact rules about what is a valid order and what is not. Also, a counter can only handle so many orders at once, but an API can take thousands per second if designed well. And crucially, a counter doesn't remember past orders — but an API can log everything for later analysis.

A model serving API is the HTTP or RPC surface that exposes a trained model to the outside world, handling incoming requests, validating their schemas, checking the model's health, and routing to the correct version of the model.

This concept builds directly on the idea of a model as a standalone artifact — something you have trained and saved. But it assumes you have already learned what a model is and how it makes predictions. From here, it leads to the rest of MLOps: you cannot do model monitoring, canary releases, or drift detection without an API to monitor. Your library has notes on Model Registry and Versioning, which will connect here: the API is what lets you switch between versions using aliases. Training-Serving Skew is also a neighbouring topic: the API is where skew often shows up because it is the boundary between training-time transformations and serving-time ones. The API is also the place where human-in-the-loop feedback gets collected, because it is the only place where real users interact with the model.