Agentic AI in Operations: Where It Works and Where It Fails

Agents are the most hyped and most misunderstood shape of AI in operations right now. The hype says agents can “do anything.” The practical reality is that agents do a narrow set of things extremely well — and burn spectacularly when pointed at the wrong shape of problem. This is a field guide to telling the two apart.

First, a definitional point that matters more than it sounds. An agent is not a chatbot and it is not a model. An agent is a system that decides, acts, and reports — usually by calling tools, usually within guardrails, and always with a trail. The model is one component. The tool-calling layer, the validation layer, the escalation policy, and the logging are the rest. When people say “agents don't work,” they almost always mean they shipped a model with a tool interface and none of the rest.

Agents shine on bounded, reversible, observable work

Agents are most reliable when three conditions hold simultaneously. The task is boundedBounded autonomybounded — the agent operates inside a clearly defined scope with explicit limits on what it can decide and do. The actions are reversible — if the agent is wrong, a human can undo the damage cheaply, or the system can roll back automatically. And the work is observable — every decision, every tool callTool callingtool call, every output is logged in a way a reviewer can reconstruct after the fact.

Good fits include triaging incoming tickets and routing them to the right queue, drafting replies for human review, extracting structured data from unstructured documents, reconciling records across two systems, and running well-defined research lookups. In every one of these, the agent is doing work that a junior person would do, the output is checked before it commits anything irreversible, and the failure mode is “a human catches it” rather than “a customer is harmed.”

Agents burn on open-ended, irreversible, unobservable work

The flip side is consistent and worth memorising. Agents burn when any of the three conditions is violated. Open-ended tasks — “research this market and tell me what to do” — burn because there is no way to bound the scope and no clear definition of done. Irreversible actions — sending money, firing an employee, deleting records — burn because a single confident mistake is catastrophic and there is no rollback. Unobservable work — the agent “reasons” internally and emits a recommendation with no trace — burns because nobody can debug what went wrong, and so the same failure repeats.

A common failure pattern is the “autonomous email agent” — a system that reads an inbox, decides what to do, and sends replies without review. It demos beautifully. In production it eventually sends something confidently wrong to an external party, and the reputational cost dwarfs whatever time it saved. The fix is not a better model. The fix is making the action reversible (drafts for review) or observable (full trace, escalation on low confidence). The model was never the problem.

An agent without an eval suite is a liability

The component that separates production agents from demos is the eval suiteEval suiteeval suite — a held-out set of inputs with known-good outputs that runs on every model change, every prompt change, and every tool change. Without it, you are deploying agents blind and discovering regressions through user complaints. With it, you catch a regression before it ships, the same way a test suite catches a bug before it reaches production.

A workable eval suite does not need to be enormous. Fifty to two hundred curated examples, drawn from real traffic and labeled with the acceptable output range, is enough to catch the vast majority of regressions. The discipline is in maintaining it — adding new examples when a new failure mode appears in production, retiring stale ones, and running the suite as a gate on every deploy. This is the unglamorous work that makes agents safe enough to ship.

The broader lesson is that agent safety is an engineering problem, not a model problem. Bounded autonomy, reversible actions, full observability, and an eval suite are the four pillars. Get all four right and agents are among the highest-leverage tools in operations. Skip any one and you are running a demo that will eventually embarrass you. The choice is yours, and it is made in the system design, not in the model selection.