Design AI systems that can use tools and complete multi-step work without giving them unlimited authority.
An agentic system repeatedly observes state, chooses an action, uses a tool, checks the result, and decides what to do next. Autonomy exists on a spectrum. A workflow can use an agent loop while still requiring people to approve consequential actions.
| Pattern | Best for | Use an agent? |
|---|---|---|
| One model call | Drafting, extraction, classification, explanation | No—simpler and easier to test |
| Fixed workflow | Known steps with predictable branches | Usually no—use deterministic orchestration |
| Bounded agent | Choosing among approved tools when the path varies | Yes, with limits and observation |
| Open-ended autonomous agent | Unclear goals, broad tools, long time horizons | Usually avoid; risk and evaluation cost rise sharply |
Use the least-agentic design that works. More autonomy is not automatically more capable, reliable, or valuable.
Define one observable success condition, non-goals, maximum duration, attempts, tokens, and spend.
Allowlist narrow functions. Prefer read-only access; isolate execution and use a dedicated identity with least privilege.
Require explicit confirmation before sending, publishing, buying, deleting, changing permissions, or exposing sensitive information.
Validate every tool argument and result. Log decisions without collecting unnecessary sensitive data. Treat retrieved content as untrusted.
Stop on success, uncertainty, repeated failure, limit reached, policy conflict, or human escalation. Make retries idempotent and preserve rollback.
Prompt instructions are not security controls. “Do not access other files” must be enforced by filesystem, network, identity, and application permissions—not trusted as text.
Search approved sources, collect passages, draft a cited summary, and stop when evidence is insufficient. A person approves publication.
Read a new ticket, retrieve policy, suggest a category and reply, then route exceptions. The agent cannot issue refunds.
Inspect a repository, propose a patch, run isolated tests, and open a draft change. It cannot merge or access production secrets.
Extract proposed actions from approved notes, check owner names, and draft tasks. A participant confirms before tasks are created.
Query read-only metrics, compare expected ranges, and prepare an incident note. It escalates rather than changing records.
Hiring decisions, medical advice, financial transfers, unsupervised public communication, or broad computer control with unclear success criteria.
Describe a candidate workflow and select its controls. The browser generates a design brief and a heuristic risk signal—not a compliance or security certification.
Test the workflow, not only the model's prose. Build representative normal, edge, adversarial, and failure cases before a pilot.
| Measure | Question |
|---|---|
| Task success | Did the system achieve the defined result with evidence? |
| Action validity | Were selected tools and arguments allowed and correct? |
| Safety | Did it resist injection, protect data, and pause at approval gates? |
| Recovery | Did timeouts, unavailable tools, and bad results lead to safe fallback? |
| Efficiency | What were total latency, model/tool cost, and human review time? |
| Reproducibility | Can logs explain what happened, and do repeated cases behave acceptably? |
Learn AI