The demo is perfect. The agent reads a request, calls two tools, and returns a confident answer in seconds. Then Monday arrives: one service is slow, a record is missing, and the agent repeats the same call until the user gives up.
That gap is not a model problem by itself. It is a product reliability problem—and it is one you can design for.
Give every task a finish line
An open-ended instruction such as “take care of this account” gives the system no useful definition of done. Replace it with a bounded outcome: “find the latest invoice and draft a reply explaining its status.” The narrower task makes success observable and limits the consequences of a wrong turn.
Set explicit budgets for tool calls, elapsed time, and retries. If a task reaches one of those limits, stop and tell the user what was completed, what failed, and what they can do next. A clear handoff is better than an invisible loop.
Treat tool calls as a state machine
For each action, track the request, result, and task state. Detect repeated calls with the same inputs, unchanged results, or a retry limit that has been reached. Make writes idempotent where possible so a network retry does not create a second order or send the same message twice.
Retries should respond to the kind of failure. A timeout may be temporary; a permission denial usually is not. A validation error needs corrected input, not another identical attempt. Classifying failures keeps “try again” from becoming the whole recovery strategy.
Make recovery visible
Show the user which step is running, when the system is waiting on another service, and when it needs help. Preserve enough task state to resume safely, but do not present partial work as if it were complete.
Before release, test the awkward cases: a tool that returns slowly, an empty result, conflicting records, revoked permissions, duplicate requests, and a user who changes the goal mid-task. Log the decisions and outcomes needed to diagnose failures without collecting secrets or unnecessary personal data.
The production-ready agent is not the one that never gets stuck. It is the one that recognizes the stuck state, avoids making it worse, and gives the user a sensible next move.
