Routing and Expert Specialization

Routing is the quiet engine behind near every modern AI service, and it directly controls three things you will care about from day one: how fast a request feels, how much it costs, and how accurate the answer is. A single giant model is slow and expensive; a set of smaller, specialised models that each handle one slice of work is faster and cheaper. Routing decides which slice each request belongs to. It also underpins the whole field of AI Agents: any agent that decides which tool to call, which document to retrieve, or which sub-agent to hand a task to is doing routing. Master this concept and a hundred later ones — agent loops, tool calling, retrieval-augmented generation, multi-agent systems — will click into place, because they all depend on the same idea: send each piece of work to the thing that does it best.

Picture a large railway station in Mumbai with a single enormous train that could stop at every station in the country. To go from Andheri to Churchgate, you would board that one train, it would chug through every stop, and you would arrive half a day later. Now imagine instead a network of local trains, each with a short route: the Harbour line, the Western line, the Central line. You pick the train that goes where you need. It is faster, it does not waste fuel hauling carriages to places they are not needed, and each train can be optimised for its own line — a slow stopping train for short hops, an express for long ones. That choice of which train to board is routing. In AI, the 'trains' are models, tools, and agents. A request — 'summarise this legal contract' — can be sent to a giant general-purpose language model, or to a small specialised model that has been trained only on legal text. The small one will be quicker, cheaper, and often more accurate at that narrow task. Routing is the decision that gets each piece of work onto the right train. It is not a single decision in most systems; it is a chain. A request arrives, a router looks at it and says 'this is a legal question', hands it to the legal agent; that agent decides it needs to look up a past case, so it calls a search tool; the tool returns a list of documents, and a second router picks the most relevant one to send back. Every one of those choices is routing, and together they are the difference between a system that feels like a magic oracle and one that feels like a well-organised office.

Think of the best hospital you can imagine. It does not have one doctor who is an expert in everything. It has specialists: a cardiologist for hearts, an orthopaedic surgeon for bones, a paediatrician for children, an oncologist for cancer. When a patient walks in, the first person they meet is not the cardiologist — it is the triage nurse. The nurse asks a few questions, checks blood pressure and temperature, and then sends the patient to the right specialist. That nurse is a router. The nurse does not need to know how to perform heart surgery; they just need to recognise which specialist is the right one for this patient. That is exactly what a routing model does in AI. It does not need to know the answer to the user's question; it needs to know which model, tool, or agent does. The analogy holds beautifully for the core benefit: specialists are better at their own domain. A cardiologist will outperform a general practitioner on a heart problem, and a legal-text model will outperform a general chat model on a contract clause. It also holds for cost: seeing a specialist is more expensive in money and time, so the triage nurse should not send everyone to the cardiologist. The same is true in AI — a 175-billion-parameter model is the cardiologist: powerful, but slow and costly. A 7-billion-parameter model is the general practitioner: fast and cheap, but less capable. The router's job is to send the easy cases to the generalist and the hard cases to the specialist. Now, where does the analogy stop working? A triage nurse in a hospital is a human with years of training; they can make nuanced judgements. An AI router is a piece of software, and its judgement is only as good as the data it was trained on. It does not understand — it patterns. It might mistake a headache that is actually a brain tumour for a migraine, because that is what the data suggested. A human nurse would probe further. Also, in a hospital the specialists are fixed; in AI you can train new specialists whenever you want, and the router can be updated to send cases to them. And finally, a hospital only has a few dozen specialists; an AI system might have thousands of tiny routers and thousands of specialised components, and the routing decision is made many times within a single conversation — not once per patient, but once per sentence, or even per token. So the analogy gives you the shape, but the scale and the fallibility are new.

Routing is the process of directing each incoming request, prompt, or sub-task to the most appropriate model, tool, or specialised agent — chosen according to some policy that balances accuracy, cost, speed, and reliability — rather than sending everything to a single general-purpose system.

This concept sits at the heart of the AI Agents parent concept in your library. An agent is a system that plans, calls tools, observes results, and iterates — and every one of those steps involves a routing decision. When the agent decides which tool to call (a calculator, a web search, a code interpreter), that is routing. When it decides which memory store to read from (working context, episodic history, semantic facts), that is routing. When it decides whether to ask a human for clarification or to proceed autonomously, that is routing. You have also noted that Agent Cards are machine-readable descriptions of an agent's capabilities — and a router is exactly what reads those cards to decide which agent to send a task to. Agent Communication Protocols route messages between agents across systems. Agent Executors route each tool call to the right handler. In your library's neighbouring topic, AI Coding Assistants, routing is what decides whether to complete a few lines or to launch a multi-file refactor. And four-axis quality — quality, safety, cost, reliability — is precisely what a routing policy weighs up: a medical diagnosis route must optimise for safety, a code-completion route for speed, a customer-support route for cost.