THE SHORT ANSWER
AI cost depends on more than a model's rate. Input and output volume, reasoning effort, tool calls, retries, latency, engineering and human review all contribute. Compare systems by the cost of a successful, accepted task—not by the cheapest request.
Choose the unit you are trying to improve
A request is a technical event. A task is the outcome someone needs: classify an enquiry, draft an approved explanation or resolve a support question. One task may require several requests, a tool call and a review. Optimising the request while ignoring the task can make the system look cheap while rework grows.
Map the full cost stack
| Driver | What changes it | What to inspect |
|---|---|---|
| Model | Capability tier, modality and provider | Does the task need this level of capability? |
| Usage | Input, output and sometimes reasoning or cached tokens | Which material is necessary for this request? |
| Operations | Tools, retrieval, retries, latency and throughput | Where does the workflow repeat or wait? |
| People | Design, evaluation, review and correction | What must a person verify, and how often? |
| Failure | Wrong answers, missed deadlines and unsafe actions | What is the consequence of an unacceptable result? |
Use one decision sequence
OpenSkool uses TASK → MODEL → CONTEXT → REASONING → OUTPUT → VERIFY → MEASURE. Define the task and acceptance rule first. Choose the least costly model that meets it. Supply relevant context, ask for only the reasoning effort and output needed, verify in proportion to risk, and measure the completed task.
The order matters. Starting with token trimming can remove evidence the model needs. Starting with the task exposes larger opportunities: a deterministic rule may replace a model call, a smaller model may handle routine cases, and only uncertain cases may need escalation.
Build a cost baseline before optimising
- Name one repeatable task and define an accepted result.
- Count requests, input, output, tools, retries and review minutes for a representative sample.
- Record quality, latency and failure consequences beside monetary cost.
- Change one driver, rerun the same sample and compare the full task.
A baseline turns cost reduction into an experiment. Without it, a lower bill may simply reflect lower volume, missing work or a quality loss that appears later.
Sources & further reading
- Models
OpenAI Developers. Official model-selection documentation, checked 13 September 2026. Product names, capabilities and prices can change; the collection uses the durable principle of matching capability to a task rather than prescribing a current model.
- Evaluate a model router
Microsoft Learn. Official guidance for evaluating routing across representative workloads using quality, cost, latency and policy criteria. It is not evidence that routing always improves results.
- Generative Artificial Intelligence Profile (NIST AI 600-1)
NIST. Risk-management guidance, including confabulation. It does not establish a universal error rate.
Examples and exercises are illustrative unless attributed to a source. No independent expert review is claimed.
A correction, a counterexample or an experience worth sharing?
Join the conversation ↗