The control pattern
Data asset
Request characteristics and runtime signals
Knobs
Complexity, Domain, Risk, Confidence, Latency and cost budgets, Tool requirements
Outcome
The right model and workflow for each request
Measurement
See how to measure this below
What it is
Model routing avoids using the same model for every task. The incoming request and its metadata are the data asset, while complexity, risk, domain, confidence and service-level requirements are the routing knobs.
A lightweight classifier can recognise the task and select a deterministic function, a retrieval workflow, a distilled student, a larger reasoning model, or human escalation. Routing is not merely cost optimisation. It improves quality by sending structured calculations to code, current-price questions to a market-data tool, and nuanced cross-document analysis to a model capable of deeper reasoning.
A strong router is also what makes distillation practical: the student handles the high-volume centre of the distribution while the teacher remains available for the difficult tail.
The knobs in detail
Each row is one adjustable property of the data asset, and what moving it tends to do.
| Knob | What you adjust | Likely effect |
|---|---|---|
| Complexity | How many documents and steps are involved | Selects between fast and deep paths |
| Domain | What kind of question is being asked | Routes to the right specialist or tool |
| Risk | Consequence of being wrong | Raises evidence and explanation requirements |
| Confidence | The student's own certainty | Triggers escalation to a teacher model |
| Latency and cost budgets | Service-level targets per surface | Keeps interactive paths fast |
| Tool requirements | Whether code or a live feed is needed | Sends calculations to deterministic components |
Applied: Stocks Assistant
For Stocks Assistant, simple ticker lookup, schema validation and standard earnings extraction can use fast specialised components. A question comparing several companies across four quarters may require richer retrieval and a larger model. A request combining options exposure, portfolio concentration, taxes and personalised action carries higher risk and should face stricter evidence and explanation requirements. The router should also detect missing context: “Should I sell?” cannot be answered responsibly without horizon, objective, position context and an explicit description of uncertainty.
How to measure it
Evidence that the knob produced the intended behaviour, rather than shifting the problem elsewhere.
- Routing accuracy, evaluated separately from answer quality
- Logged intent, chosen route, confidence, tools called and fallback reason
- Latency and cost per route
- End-to-end answer quality by route
Common mistakes
Evaluating only the downstream answer. A perfect model cannot rescue a misclassified request.
Unversioned routing rules, which make regressions impossible to attribute.