Start with the existing work
Map the trigger, inputs, decision, output, reviewer and consequence before inserting AI. Remove steps that exist only because information was previously hard to transform. Preserve controls that exist because a mistake matters.
Distinguish a predefined workflow from an agent that decides its own next steps. More autonomy can help variable work, but it also expands the paths, tools and failures that need observation.
Evidence & context: Anthropic
Use a staged pattern
| Stage | Efficient default | Escalate when |
|---|---|---|
| TASK | Check eligibility with rules | The task is ambiguous or out of scope |
| MODEL | Use the smallest validated option | Confidence or category requires more capability |
| CONTEXT | Retrieve a small evidence set | Required evidence is missing or conflicting |
| REASONING | Use bounded effort | Complexity is real and measurable |
| OUTPUT | Return a schema the next step uses | A person needs explanation or alternatives |
| VERIFY | Automated checks plus risk-based review | A check fails or consequence is high |
| MEASURE | Log task result and resource use | Drift or repeated failure appears |
Make limits part of the design
- Set maximum attempts, tool calls and elapsed time.
- Cache stable material only where the provider and workload support it.
- Batch independent work when latency requirements allow.
- Use idempotency or deduplication so retries do not repeat external actions.
- Return a clear failure state instead of inventing completion.
- Give people the evidence needed to approve or correct the result.
Evidence & context: OpenAI Developers · NIST
Evaluate the workflow end to end
Model accuracy alone misses routing errors, retrieval failures, tool downtime and reviewer burden. Replay representative tasks through the complete path. Measure category-level quality, total latency, cost per accepted task, escalation rate and severe failures.
Complex routing is justified only when the observed savings or quality improvement exceeds its maintenance and failure cost. Begin with a legible workflow and add branches from evidence.
Evidence & context: Microsoft Learn
Sources & further reading
- Building effective agents
Anthropic. A provider's engineering taxonomy of agents and workflows, not a universal industry definition. We use the conceptual distinction, not its changing product recommendations.
- Evaluate a model router
Microsoft Learn. Official guidance for evaluating routing across representative workloads using quality, cost, latency and policy criteria. It is not evidence that routing always improves results.
- Prompt caching
OpenAI Developers. Provider-specific guidance on reusing repeated prompt prefixes to reduce latency and input cost. Availability and implementation details vary by model and provider.
- Generative Artificial Intelligence Profile (NIST AI 600-1)
NIST. Risk-management guidance, including confabulation. It does not establish a universal error rate.
Examples and exercises are illustrative unless attributed to a source. No independent expert review is claimed.
A correction, a counterexample or an experience worth sharing?
Join the conversation ↗