What Is Model Routing?
Model routing is the practice of putting a decision in front of a set of configurations, so that each incoming request is dispatched to one of them by a written policy rather than to a single fixed choice. The decision is taken by ordinary code before the request is sent, from properties of the request that exist at that moment, and its output is a name: which configuration handles this one.
Where the decision is made, and what it is allowed to read
A router reads what exists before dispatch, which in practice is a short list: a declared task class or endpoint, the length of the input, whether tools are required, the latency the caller can wait, what the request is permitted to cost, which tenant it belongs to and where their data may be processed, and whether this is a first attempt or the retry of something that already failed. Conspicuously absent from that list is the answer, which is why routing is always a decision about a request and never a decision about a result.
Three policy shapes cover most deployments. A static table maps request features onto a destination; it is the cheapest to run and the only one that can be read as a document. A learned router puts a small classifier in front, predicting which destination will suffice, and it adds a stage that is itself sometimes wrong and whose latency is paid on every request, including the ones a table would have routed correctly. A cascade inverts the question entirely: send the request to the cheaper destination, apply a check to what comes back, and escalate only when the check says so — which converts a prediction problem into an evaluation problem, and is affordable only where the check costs less than the escalation it avoids.
Where the decision physically lives then decides who can audit it. In the application, the policy is source in the caller's own repository, changes pass through review, and the destination can be logged beside the result. In a gateway or a proxy, one policy serves many callers and none of them can see it from their own code. In a provider-side product, the decision is taken on the far side of the request and the caller learns which destination served them only if the response says so. All three are legitimate arrangements; they differ in whether the decision that produced a given output is recoverable afterwards, and that difference is the one that matters on the day an output has to be explained.
| Policy | What it costs and what it makes visible |
|---|---|
| Static rules over request features | Cheapest to run and readable as a document. It cannot react to anything the request does not carry. |
| Length or budget thresholds | Trivially auditable. The threshold is a constant somebody chose, and it is right only for the traffic it was chosen against. |
| A classifier in front of the dispatch | A stage on every request, paid in latency, plus a second component that can be wrong in a way of its own. |
| Cascade with an escalation check | A second call on the escalated share, and a check that has to be both cheaper and more trustworthy than the escalation. |
| Fallback on error or unavailability | Nothing until it fires. When it fires, the destination changes without the caller asking, which is why the decision belongs in the log. |
The rows share a property worth naming: each one makes the destination a function of something other than the caller's intent. That is the point of routing and also its bill. A system with a router in it does not have one behaviour; it has as many as the router can select between, and the number it can exhibit is the number of destinations multiplied by the conditions that can reach them.
What routing does to everything downstream of it
It removes uniformity, and the loss shows up first in reproduction. Two identical requests can be served by different destinations whenever the policy reads anything time-varying — load, availability, a retry flag, an error fallback — so an output can no longer be explained by the request alone. The repair is procedural and cheap: record the routing decision with the result, as a field, at the moment it is taken. Without it, a change in output looks like drift in a configuration that was never involved.
It fragments everything that depends on a stable destination. A cached prefix belongs to the configuration it was built against, so splitting traffic splits the cache with it. Prompts are not portable unchanged either: format instructions, tool schemas, stop conditions and worked examples were written against one destination, and a router either carries a variant per destination — more artefacts to keep in step — or accepts that one set of instructions is being sent to destinations it was not written for. Evaluation splits the same way, since a single aggregate over routed traffic averages across destinations and conceals which one moved; a routed system is measured per route and then end to end, not instead of it.
And it introduces a failure belonging to the router rather than to any destination. A policy is a set of conditions, and the requests satisfying none of them, or two at once, have to go somewhere: the default destination is a decision, and it is usually taken by whoever wrote the last branch. Meanwhile the traffic keeps moving. A threshold chosen against one month's inputs sends a different share of the next month's the other way, with nothing in the system having changed and nothing reporting that anything has. This is why a router is monitored on the distribution of its own decisions and not only on the outcomes of the requests it dispatched — the distribution moves first.
How the Registry classifies a request routing task
Dispatching each inbound request to one of several configurations according to a written policy is filed as data processing in this registry's classification. The class reads the operation the task itself performs: properties of a request are inspected, a rule is evaluated over them, and what the task produces is a derived label — a destination — rather than any of the work that destination will subsequently do.
The Registry's canonical brief for this entry is filed as:
Dispatch each inbound request to one of several configurations according to a written routing policy.
Submitted for assessment it is
classified as Data processing, and its wording is hashed once — to
6e3ddf588db45d01…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/analysis.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about model routing
Which configurations sit behind a router, what any of them costs, and which of them suits a given request are outside anything this page or the canonical brief above establishes. This registry operates no router, compares no configuration against another, and publishes no figure for the cost or the quality of any of them. Its own derivation reads the configuration named on an order and does not select one: a submitted model name, training cutoff and temperature are the inputs, and the method contains no branch that could substitute a different set.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about model routing
- What is model routing?
- Model routing is a decision layer in front of several configurations that dispatches each request to one of them by a written policy. The decision is taken by ordinary code before the request is sent, and it reads properties of the request rather than anything about the eventual answer.
- What does a router read to make its decision?
- Whatever exists before dispatch: a declared task class, the input length, whether tools are needed, a latency budget, a cost ceiling, tenancy or data-residency constraints, and whether the request is a retry. It cannot read the result, because the result does not exist yet.
- What is a cascade router?
- A cascade sends the request to the cheaper destination first, applies a check to what comes back, and escalates to a more expensive one only when the check fails. It replaces a prediction problem with an evaluation problem, and it is worth doing only where the check costs less than the escalation it avoids.
- Why should a routing decision be logged?
- Because without it an output cannot be attributed. A router makes the destination a function of conditions outside the request, so two identical requests may be served differently. Recording the destination alongside the result is what stops a later investigation from looking like unexplained drift.
- Does routing affect prompt caching?
- Yes. A cached prefix belongs to the configuration it was built against, so traffic split across destinations is cache locality split across them too. The same fragmentation applies to prompts, tool schemas and evaluation sets, each of which was written or measured against one destination.
Related entries
Order an assessment for data processing tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.