Engineering the Agent Tool Loop

An agent is a controlled state machine

The familiar loop—observe, reason, act—becomes reliable only when each transition is explicit. Represent user intent, messages, tool results, pending approvals, budgets, and terminal states in durable structured state. The language model proposes the next action; application code validates and executes it. This boundary matters because a model response is untrusted input, even when it looks like a well-formed function call. The orchestrator owns truth about what happened.

Define terminal conditions before enabling tools. Success should correspond to a verifiable state, such as a created artifact or reconciled record, not the model claiming completion. Other terminal states include user cancellation, deadline, exhausted budget, denied permission, and unrecoverable dependency failure. Without explicit endings, agents repeat searches, retry impossible calls, or produce a confident summary after a tool silently failed.

Keep model context as a view derived from state rather than the state itself. Conversation transcripts can be summarized or truncated; authoritative tool outputs, identifiers, and approval decisions cannot. Store large results externally and pass the model bounded references or selected fields. A deterministic state reducer makes runs replayable and lets tests inject recorded observations without calling real systems. It also prevents prompt formatting changes from corrupting operational history.

Tools need narrow contracts

A tool schema should expose the smallest meaningful capability. A generic execute-command or arbitrary-SQL tool gives the model an enormous failure surface. Prefer createInvoiceDraft, searchApprovedDocuments, or scheduleWithinPolicy, with typed parameters and server-side authorization. Include field descriptions, constraints, side effects, and idempotency behavior. Tool names and descriptions are part of model performance, but runtime validation is the security boundary.

Separate read actions from writes and reversible writes from consequential actions. The orchestrator can freely retry idempotent reads, but sending a payment or publishing content may require preview and explicit approval. Return stable result codes rather than burying errors in prose. A model can reason more reliably about NOT_FOUND, RATE_LIMITED, and NEEDS_APPROVAL than an inconsistent stack trace. Preserve human-readable details for diagnosis without asking the model to infer control flow from them.

Tool results should contain only what the next decision needs. Dumping a thousand-row response increases cost and makes malicious data more likely to enter context. Add pagination, server-side filters, and summary fields, while retaining a handle for deeper inspection. Mark provenance and trust level, especially for web pages or user-generated content. Retrieved text can inform the task but must never rewrite tool policies or impersonate system instructions.

Budgets and plans prevent wandering

Every run needs budgets for model tokens, tool calls, wall time, monetary cost, and possibly external side effects. Budgets are enforced by the orchestrator, not merely mentioned in the prompt. Reserve capacity for the final response and recovery. A loop that discovers exhaustion only after spending the last token cannot explain partial progress. Expose remaining budget to the model in a compact control block so it can choose a cheaper path.

Plans are useful when they are short, revisable, and connected to execution state. A twenty-step narrative created before observation becomes stale immediately. Track objectives, active step, dependencies, and verification criteria. After a surprising tool result, re-plan explicitly rather than continuing a fictional sequence. For simple requests, skip planning overhead and execute one validated action. The architecture should support both paths instead of forcing every interaction into agent theater.

Loop detection compares recent actions, arguments, and observations. Repeating the same failing call with unchanged inputs should trigger a recovery strategy or stop. Semantic loops also matter: the agent may alternate equivalent searches phrased differently. Use bounded histories and counters by failure class. Escalate to the user with the exact missing decision or unavailable dependency, not a generic request for guidance that transfers diagnosis back to them.

Verify effects, then explain

A successful tool response does not always prove the intended outcome. After a write, read back the affected object or use a receipt carrying its canonical identifier and version. For code changes, run focused tests; for scheduling, confirm the stored timezone and participants; for deployment, inspect health. Verification steps should be designed with the same specificity as execution. Asking the model whether it thinks the action worked is not verification.

Retries require idempotency keys derived from the logical action, not a new random value each attempt. Persist the key before calling an external service so process crashes cannot create duplicates. Classify errors into retryable, correctable, approval-dependent, and terminal. Exponential backoff belongs to transient infrastructure failures, while invalid arguments should return to the model with structured constraints. Never retry a consequential unknown result until reconciliation proves whether the first call committed.

The final response should be generated from verified state and distinguish completed actions, skipped actions, and unresolved items. Include user-facing identifiers and recovery options without exposing internal chain-of-thought or sensitive logs. This closes the loop between execution and trust: the user sees what changed, what evidence confirms it, and what still needs attention. Reliable agents feel less magical precisely because their claims correspond to inspectable effects.

Research foundation and scope

ReAct demonstrated an interleaving of reasoning and external actions, while Toolformer explored learning when and how to call tools [REACT] [TOOLFORMER]. In a production application, deterministic orchestration must own permissions, state, side effects, and terminal conditions. These sources establish the mechanism, but they do not remove the need to measure the local implementation. A paper's result belongs to its checkpoint, dataset, hardware, traffic, and experimental protocol; a standard describes a contract or risk practice, not a guarantee that a particular product follows it. The useful engineering move is to convert each cited idea into a hypothesis that can be falsified with representative inputs and observable intermediate state.

Model text is untrusted input even when it conforms to a tool schema. A tool proposal never authenticates a principal, proves an effect, grants approval, or establishes that the user's goal was completed. Write these conditions into the design review before selecting a library or provider. Specify the authoritative source of truth, the authenticated principal, the versions that influence behavior, the maximum consequence of an error, and what the system will do when required evidence is missing. This boundary statement makes later optimization honest: teams can compare speed, cost, or convenience only after candidates satisfy the same correctness, authorization, privacy, and recovery contract.

Read [REACT], [TOOLFORMER], [NIST-GAI] together rather than treating one source as a recipe. ReAct: Synergizing Reasoning and Acting in Language Models supplies one part of the foundation; Toolformer: Language Models Can Teach Themselves to Use Tools supplies a second perspective; Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile helps connect the mechanism to evaluation, governance, or operations. The citations are deliberately primary papers, standards, or official documentation. Product-specific claims still require local evidence, and any dated behavior must be rechecked against the cited version before an article or architecture decision is refreshed.

A production implementation blueprint

Represent runs as explicit states, expose narrow typed tools, validate arguments server-side, separate reads from writes, require approval at consequence boundaries, issue idempotency keys, and store receipts independently of model context. Begin with one thin vertical path that preserves all decision evidence. Give each run a stable identifier; capture normalized input, authenticated scope, relevant dependency versions, selected policy, timing, result status, and any external receipt. Keep large or sensitive payloads in a separately governed store and pass bounded references through the orchestration layer. This produces a debuggable system without making raw customer content the default telemetry substrate.

Make state transitions explicit and make every external boundary return a typed result. Distinguish validation failure, permission denial, missing evidence, rate limit, timeout, dependency outage, user cancellation, and internal defect. The distinction matters because each permits a different response: clarify, retry, fall back, defer, compensate, or stop. A generic exception handler that asks the model to try again turns uncertainty into repeated cost and can duplicate irreversible effects. Retries require an idempotency strategy and a stored receipt, not optimism.

Roll out the implementation as a versioned policy. Record which cohort receives it, what baseline it replaces, which metrics can stop the rollout, and how to reverse stateful artifacts such as indexes, caches, adapters, or workflow histories. Review privacy and retention alongside performance. If raw content is needed temporarily for diagnosis, state the purpose, access, sampling, encryption, and deletion window. Production readiness means the team can explain, contain, and reverse behavior under load—not merely demonstrate the happy path once.

Failure lab and evaluation plan

Replay recorded observations through the state reducer, fuzz tool arguments, deny permissions, reorder results, and compare plans under fixed budgets. Evaluate completion against external state rather than model self-reports. Keep the arrival pattern, data distribution, authorization, and dependency versions close to production. Use a paired baseline where possible and retain per-case results so averages cannot hide a severe slice. Offline evaluation should localize the failing stage; shadow evaluation should expose integration and traffic effects; a canary should confirm user-facing outcomes with a small blast radius. Each stage needs an exit criterion rather than a vague request to 'look good.'

Simulate tool timeout after a committed write, stale observations, malicious retrieved instructions, pagination loops, expired approval, repeated search, user cancellation, and a provider switch mid-run. Verify safe resume or explicit termination. Add adversarial boundary cases and partial failures, not only malformed inputs. Terminate a worker after an external system may have committed, revoke access after a cache was populated, change one dependency version, and delay one branch until its data is stale. Confirm that cancellation releases resources and that a fallback preserves semantics. The test passes only when the external outcome, audit evidence, and user-visible state agree; a fluent final message is not proof of completion.

Evaluation data needs lineage equal to production data. Record why each case exists, who reviewed the expected behavior, which source or policy supports it, and when it must be refreshed. Separate generated cases from production incidents and prevent test cases from leaking into training. When a metric changes, inspect changed examples before accepting an aggregate. A statistically small regression can still block release if it crosses an authorization, privacy, safety, or irreversible-action boundary.

Operating metrics and decision record

Track goal completion, tool success and validation rates, repeated actions, approval conversion, steps and cost per outcome, time spent waiting, rollback use, side-effect duplicates, and human correction. Define every metric with a numerator, denominator, unit, time window, exclusions, and responsible owner. Segment by workload and consequence so a dominant easy class cannot hide a rare costly failure. Pair service signals with product outcomes: latency without completion encourages fast useless answers, while answer quality without queue and cost data can produce an uneconomic service. Review percentiles and distributions rather than relying on a single average.

Grant the model only proposal authority; application code decides whether an action is authorized, affordable, current, idempotent, and proven complete. Put that choice in a short architecture decision record containing context, alternatives, evidence, assumptions, selected policy, rollout, reversal trigger, and unresolved risk. Link the evaluation snapshot and source versions. This document prevents a benchmark from becoming folklore and tells the next engineer which evidence must be repeated when the model, provider, traffic, data, regulation, or product promise changes.

Monitor leading and lagging signals after release. Leading signals reveal pressure—queue age, cache allocation, disagreement, validation failure, policy denial, or retry growth—before users report harm. Lagging signals show whether the job was actually completed and trusted. Alert on a service objective tied to user consequence, and make the run identifier available to support without exposing private content. Every serious miss should become a reproducible regression case and, when relevant, a new threat or data-quality control.

Primary sources and further reading

1. ReAct: Synergizing Reasoning and Acting in Language ModelsOriginal research paper · paper
2. Toolformer: Language Models Can Teach Themselves to Use ToolsOriginal research paper · paper
3. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileNational Institute of Standards and Technology · standard