Compare the system, not one adjective

Typical trade-offs that require task-level testing
QuestionSmaller model may suitLarger model may suit
Task structureNarrow, repeated, well specifiedAmbiguous, varied or multi-step
InputsStable formats and vocabularyMessy context or nuanced instructions
OperationsHigh volume or strict latencyLower volume where difficult cases dominate
RiskEasy to validate or safely rejectComplex judgment, still with appropriate verification

These are hypotheses, not universal laws. Architecture, training, tooling and task fit can make a smaller model outperform a larger one on a bounded evaluation.

Include the work around the model

A smaller model that needs elaborate prompts, repeated retries and constant manual correction may cost more per accepted task. A larger model that completes a difficult task once may be more efficient. The reverse is equally possible when an expensive model handles a simple classification that a bounded method could solve.

Use a capability portfolio

Instead of declaring one organisation-wide winner, map task classes to the simplest reliable method. Deterministic code can enforce rules, a small model can handle routine language variation, and a more capable model can receive ambiguous cases. Human escalation remains part of the design where consequences require judgment.

Keep the routing explanation simple enough to observe. If no one can tell why a case escalated or failed, savings will be difficult to defend.

Evidence & context: Microsoft Learn

Run a blind comparison

  • Create representative cases and an acceptance rubric.
  • Remove model names from outputs during review where practical.
  • Record pass rate, severe failures, latency and full task cost.
  • Inspect results by task category, not only as one average.
  • Choose a fallback and a date to re-evaluate.

Sources & further reading

  1. Models

    OpenAI Developers. Official model-selection documentation, checked 13 September 2026. Product names, capabilities and prices can change; the collection uses the durable principle of matching capability to a task rather than prescribing a current model.

  2. Evaluate a model router

    Microsoft Learn. Official guidance for evaluating routing across representative workloads using quality, cost, latency and policy criteria. It is not evidence that routing always improves results.

Examples and exercises are illustrative unless attributed to a source. No independent expert review is claimed.

A correction, a counterexample or an experience worth sharing?

Join the conversation ↗