Your AI agent works in dev. It demos beautifully. It impressed the board. And it will break in production — probably within the first two weeks of real traffic.
The frustrating part is that the failure modes are predictable. We've seen the same twelve patterns kill agents across fintech, regulated industries, and enterprise rollouts. They're "silent" because none of them produces a clear error message. They produce wrong answers, slow answers, expensive answers, or answers that look right but aren't.
This is the field guide for the twelve we keep seeing, and the fix for each.
Why these are silent
Most production failures in traditional software are loud. A service crashes, a request times out, a database throws an error. Pagers fire. Engineers respond.
Agent failures are quieter:
- The model returns confident text that's subtly wrong
- A retrieval pipeline returns documents that pass relevance checks but don't actually help
- A tool gets called 47 times instead of 1, and the bill arrives two weeks later
- An evaluation suite continues to pass while real users are getting bad answers
There's no exception to catch. The system worked. It just didn't work correctly. That's what makes these killers silent.
The twelve
01. Tool Definition Bloat
What it looks like: The agent has access to forty tools. Each request takes 8x longer than it should. The model frequently picks the wrong tool.
Why it happens: Every team adds "just one more tool" to their agent. The schema gets shipped to the model on every request. The agent has to reason about which tool to use, and reasoning degrades as options multiply.
Fix: Expose fewer, sharper tools with clear contracts. Treat the tool set as a product surface area, not a junk drawer. Use a router pattern: a small set of meta-tools that fan out to specialized sub-agents, each with a narrow tool list.
02. Context Window Decay
What it looks like: The agent follows instructions for the first five turns of a conversation, then drifts. Critical rules from the system prompt stop being enforced.
Why it happens: As the conversation grows, the system prompt gets buried under user messages and tool outputs. Models pay disproportionate attention to recent context.
Fix: Re-anchor critical rules deliberately. Compress conversation history into summaries. Inject the most important constraints into the most recent turn, not just the original system prompt.
03. Retrieval Poisoning
What it looks like: Your RAG pipeline retrieves the top-K chunks, and the agent answers with 100% confidence — even when the retrieved chunks are tangentially related or outright wrong.
Why it happens: Embedding similarity is not relevance. Two chunks can be semantically close to the query and miss the actual answer entirely. Models don't know the difference.
Fix: Filter, rank, and verify retrieved chunks before use. Add a relevance check step. Make the model cite specific chunks so you can audit grounding. Most importantly: measure retrieval quality independently of agent answer quality.
04. Runaway Agent Loops
What it looks like: An agent enters a loop. Call → retry → error → call → retry. The bill for that one request is $47.
Why it happens: Agents are designed to try harder when something fails. Without loop guards, "try harder" becomes "try forever."
Fix: Set budgets per request (max tokens, max tool calls, max wall-clock time). Define stop conditions explicitly. Treat the loop counter as a first-class metric in your observability.
05. Silent Schema Drift
What it looks like: Last week, the API returned user_id. This week it returns userId. The agent didn't crash — it just started silently mishandling every response.
Why it happens: Agents are tolerant. They'll parse around small changes, often without flagging them. Schemas evolve without coordination across the team that owns the data and the team that uses it.
Fix: Version schemas explicitly. Validate every boundary. Test agents against contract changes before they ship. Treat the data your agent consumes as you'd treat any other production API.
06. Eval Blindness
What it looks like: Your evaluation suite is green. Real users are unhappy. Investigation reveals the eval set is six months old and looks nothing like current traffic.
Why it happens: Evals get built once, then forgotten. Production traffic evolves. The eval set ossifies.
Fix: Evaluate on real traffic slices, not toy samples. Rebuild your eval set quarterly from production interactions (with appropriate consent and PII handling). Treat eval quality as a first-class metric, equal to model quality.
07. Hidden Non-Determinism
What it looks like: The same input produces different outputs across runs. Sometimes the agent is correct, sometimes it isn't. Reproducing the bad path is nearly impossible.
Why it happens: Model sampling, retrieval randomization, tool ordering, parallel execution — every layer introduces non-determinism. Even with temperature=0, providers can change underlying model weights between requests.
Fix: Control randomness where you can — pin seeds, freeze model versions, log every random choice. Trace why outputs diverge. Accept that some non-determinism is structural, and design your tests to handle it.
08. Cost Blind Spots
What it looks like: Your AI budget for the quarter was $20K. You're at $48K by week six. Nobody can tell you where the spend went.
Why it happens: Cost is emitted at the API call level by providers, but not at the business value level. You don't know which use case, which team, or which customer drove the spend.
Fix: Instrument token, tool, and latency cost per task, not just per API call. Tag every request with use case, team, and customer (where applicable). Aggregate in a dashboard your CFO can read. Set alerts at the use case level, not just the account level.
09. No Failure Mode
What it looks like: The agent is asked something it doesn't know how to answer. Instead of refusing, it fabricates an answer that sounds plausible. Users believe it. Damage spreads silently.
Why it happens: Models default to producing fluent text. Refusing is a learned behavior that requires explicit design.
Fix: Give agents safe fallback paths and refusal behaviors. Make "I don't know" a first-class output. Train evaluators to catch fabrication. Most importantly: design the UX to handle refusal gracefully, so users prefer it over fabrication.
10. Data Residency Surprises
What it looks like: An agent serving EU users calls a model hosted in US-East. Three months in, a GDPR audit flags the data flow. Launch gets paused.
Why it happens: Model providers route to wherever capacity exists. Without explicit region pinning, your data goes wherever the load balancer decides.
Fix: Pin model providers to specific regions per workload. Audit every gateway for cross-border data flows. Build region-awareness into your LLM Gateway so it's enforced at the architecture level, not at each call site.
11. Compliance Review Derailment
What it looks like: Your agent is technically ready. The model has been validated. Then compliance asks "can you explain why the model recommended denying this loan?" — and you can't. Launch is delayed by months.
Why it happens: Most agent architectures are designed for capability, not auditability. The chain of reasoning, the inputs, the tools called — all of it is needed for explanations, but rarely captured cohesively.
Fix: Design for explainability from day one. Capture full execution traces — prompts, retrievals, tool calls, reasoning steps, output. Make this auditable and queryable. In regulated industries, this layer is not optional; it's the gate that decides whether you ship.
12. Single-Vendor Failure Mode
What it looks like: Your model provider has an outage at 9am Tuesday. Your entire AI surface goes down. Customer support floods. The CTO learns about the failure from Twitter.
Why it happens: Convenience. Building against one provider's SDK is faster than building an abstraction. The abstraction feels like over-engineering — until the outage hits.
Fix: Build a vendor abstraction layer (the LLM Gateway from our reference architecture). Identify a fallback provider for each use case. Test failover quarterly, not when it's needed. Accept that no single provider is reliable enough to be your only one.
The pattern
Read these together and a theme emerges: most agent failures come from over-trust. Over-trust in the model's tool selection. Over-trust in retrieval relevance. Over-trust in the eval suite. Over-trust in a single provider's uptime. Over-trust in "the agent will figure it out."
The fix, in every case, is the same: assume the agent will fail in this specific way, design the surrounding system to catch it, and make the failure observable.
That's the entire discipline of production AI in one sentence. The architecture is just the scaffolding to make it possible.
—
If you suspect your agents are exposed to several of these and you don't know which, our AI Production Readiness Audit maps your specific exposure in two weeks with a remediation plan. Or start a conversation and we'll figure out the shape together.