Route models by task risk

Separate capability from consequence

Routing asks two questions: how difficult is the reasoning, and what happens if the output is wrong? A repetitive fixture update may require broad repository context but have low consequence because focused tests and a diff expose mistakes. A two-line authentication change may be easy to describe but high consequence because a subtle error changes who can access data. Build a task record with ambiguity, reversibility, blast radius, data sensitivity, tool authority, and available automated checks. Then select the least costly evaluated configuration that meets the required acceptance rate and review burden for that class. Official model catalogs can describe relative positioning and capabilities, but they do not establish performance on a team's repository or risk tolerance.

Use lanes with escalation triggers

A practical policy can use three lanes. Routine tasks are narrow, reversible, well tested, and handled by a lower-cost model with strict scope. Standard engineering tasks have cross-file reasoning or moderate ambiguity and use a stronger default. Critical tasks affect identity, money, destructive data operations, production policy, or an incident; they require the strongest evaluated workflow plus expert review, and some remain human-only. Escalation should happen when evidence changes: repeated tool failure, conflicting sources, tests that cannot run, scope expansion, sensitive paths, or low-confidence diagnosis. Do not let the current model decide alone that it deserves more authority. Escalating capability may help reasoning, but permissions and approval gates remain fixed unless a human changes the contract.

Worked example: a hypothetical routing table

Imagine a hypothetical team with three evaluated configurations. Its low-cost lane may update generated API examples when the schema diff is machine-checkable. Its standard lane may implement a small endpoint behind an existing authorization pattern. Its critical lane may analyze, but not autonomously apply, a session-token rotation. If the example update touches handwritten authentication code, routing stops rather than silently escalating. If the endpoint agent cannot execute contract tests, the task returns incomplete. For token rotation, the model can inventory call sites and draft a runbook, while credential creation, production changes, and revocation require named operators. This policy routes both model strength and permitted outcome; calling the most capable model would not make an unattended credential operation acceptable.

Calibrate with accepted outcomes

Collect per-lane evidence: first-attempt acceptance, reviewer minutes, correction rounds, escaped defects, refusal or clarification quality, latency, and usage cost where available. NIST's AI risk framework organizes work around governing, mapping, measuring, and managing risks; use that lifecycle to keep routing tied to the actual application. Promote a task class to a cheaper lane only after representative trials meet a stated threshold and reviewers agree the evidence remains adequate. Demote or suspend a lane after repository architecture, model, scaffold, tool permissions, or test reliability changes. Avoid hard-coding volatile product prices into the policy. Store model and configuration identifiers with every result so cost and performance can be recomputed from current records rather than remembered marketing claims.

NIST AI Risk Management Framework ↗

OpenAI model selection overview ↗

Audit routing overrides

Review manual lane changes separately from ordinary outcomes. Frequent promotion can reveal a task class whose ambiguity was understated; frequent downgrade can show that the default is needlessly expensive. Require a short reason code so overrides improve the policy instead of becoming invisible exceptions that defeat calibration.