Multi-Tenant Adapter Serving

This concept is the bridge from fine-tuning to deployment. If you've fine-tuned a model for a client or your own business, you now need to serve that model to many users without buying a separate GPU for each one. Multi-tenant adapter serving lets you pay for one base model but serve hundreds of customised versions of it. Mastering this unlocks: Inference Optimization, LLM Evaluation, and the practical skills of building a model-serving platform. You'll also see why PEFT exists — this is the deployment-side payoff of LoRA and QLoRA. Without this, your fine-tuning work stays on your laptop and never reaches a customer.

Imagine you run a small restaurant with a single popular dish — a base curry recipe. Your regulars all come in and order it, but each one tweaks it: extra chilli for Ravi, no onions for Priya, extra garlic for Mrs. Shah. You don't cook each bowl from scratch — that would mean reheating the whole pot each time. Instead, you keep the base pot simmering all day, and when a customer orders, you quickly add the individual extras from small jars you've kept aside. The base curry is the 'base model'. The individual add-ons — chilli, no onions — are the 'adapters'. You serve each customer by combining the base with their specific adapter at the moment of service. The same pot, the same kitchen, but each plate is customised. In the AI world, 'multi-tenant adapter serving' means: keep one large language model loaded in memory all the time, but for each incoming request, load the small adapter belonging to that particular customer, and run the model using that adapter. The heavy lifting (the base model) is done once; the customisation is light and fast. The economics are the whole point: you don't need a separate GPU for each customer's fine-tuned model, you just need the base plus a small adapter that slots in and out per request.

Think of a hotel kitchen. The hotel has one main kitchen (the base model) and a set of specialised chefs who can be called in (the adapters). One day, a guest from Japan requests kaiseki; the next hour, a vegetarian Italian asks for a risotto. The kitchen doesn't rebuild itself each time; the head chef (the base) stays, and a specialist chef (the adapter) joins for the duration of that guest's meal, guiding how the food is prepared. The cost of the kitchen is shared; the specialist is only hired for the hour. This works because the specialist is small and quick to bring in, and doesn't replace the whole kitchen. Now, where this analogy breaks down: in the hotel, bringing in a specialist might require a phone call and a few minutes. In AI, the swap must be nearly instantaneous — a few milliseconds — because requests arrive one after another. Also, in a real kitchen, the specialists might clump together and slow things down if guests arrive at the same time; in adapter serving, the serving system must manage the queue carefully to avoid delays. And critically, in a hotel, one specialist could accidentally move pots or leave ingredients that poison the next guest's meal — that's the security isolation issue: adapters must be completely sealed off from one another, so one tenant's data or behaviour never leaks into another's service.

Multi-tenant adapter serving is the practice of running a single shared base model in memory and, for each incoming request, loading the small, task-specific adapter belonging to the requesting tenant so that the combined model produces customised outputs, with the swap fast enough to happen per request and with strict isolation between tenants.

In your notes, this sits under Fine-Tuning. It directly builds on PEFT and LoRA and QLoRA: those are what make adapters small enough to swap per request. It also relates to Inference Optimization and Post-Training Alignment — serving is where all that fine-tuning finally meets real users. If Catastrophic Forgetting is a risk in training, adapter serving avoids it by keeping the base frozen — the adapters carry the domain knowledge, and the base never forgets. This is what the book 'Fine-Tuning with Python' calls 'the economics of serving' — one base model serving many tenants is the reason PEFT exists as a deployment strategy.