The Failure No One Is Watching
I once followed a map app’s directions for forty minutes to reach an art fair and arrived at a construction site. The venue had closed half a year earlier. The app routed me there with complete confidence, no hesitation, no note that the address had not been verified in months, no hint that the answer it was handing me had quietly gone stale. It gave me a fluent, plausible, wrong answer, and the only reason I ever found out was that I ended up standing in an empty lot. That is the consumer-scale version of the most expensive thing that happens to agentic products in production, and the reason it is so expensive is that in production there is no empty lot to stand in. The agent keeps giving the same kind of answers, and nobody arrives anywhere to discover it.
This is where the book crosses the line it drew at the start. The previous part was about how AI reshapes the work of building, the half that every team gets, the agentic product and the conventional one alike. From here on the subject is the thing that only the agentic product has: a product that acts on its own after it ships, and the failures that come with it. The crafts converging was the easy half. What follows is the half that has no equivalent when the thing you built just sits there and waits to be operated, because this thing does not wait. Before the failures begin, one idea frames all of them, and it is not new, which is the point. Forty years ago James Reason drew the distinction between active failures, the visible error at the sharp end, and latent conditions, the quiet flaws built into a system that lie dormant until an active failure lines up with them, and he gave it the image the safety-literate reader already trusts, the holes in slices of Swiss cheese that do harm only when they momentarily align. Every failure in this part is a latent condition: the over-scoped token, the unstaffed gate, the expired dataset, the unowned seam, each one harmless in isolation, each one waiting for the day the holes line up and a real case falls through. The seam thesis of this book is Reason’s argument rediscovered for agents, and naming it here buys the rest of this part a four-decade pedigree across every high-consequence field that learned the hard way that the error at the sharp end is rarely the place the system actually broke.
Eighteen months after a system ships is the moment it is most likely to be quietly broken and least likely to be caught, and the reason is the cleanest argument in this book for why an agentic product cannot belong to one person.
Start with what the failure looks like, because it does not look like a failure. A deployed agent keeps producing fluent, plausible output. No error is thrown. No alarm fires. The team’s dashboards stay green, because the dashboards measure what they measured at launch and the agent is still doing that. And underneath, the agent’s real performance has been sliding for months, in a direction no instrument on the team is pointed at, because the instruments were built to catch the failures the team imagined at launch and this is not one of them. The clinical version of this is documented and it is sobering: a hospital sepsis-prediction model deployed across hundreds of hospitals carried a marketed accuracy score in the high seventies to low eighties, and an external academic team, years into the deployment, measured its real discrimination at sixty-three, with sensitivity around a third. The number that matters is not sixty-three. It is the years. A patient-safety-critical model ran in a regulated industry, well below its advertised performance, for that long, before a single independent party, working out of curiosity rather than any compliance trigger, ran the number that revealed it. No regulator required the check. No vendor volunteered it. The model was producing fluent, plausible, wrong output the whole time, and the system that should have noticed did not exist.
Call the condition silent degradation: a deployed agent producing confident output while its actual performance decays across several axes at once, none of them caught by the instruments built at launch. Silent because nothing breaks. Degradation because the trend is downward even while the team experiences stability. It has no settled name in product management, which is part of why most enterprise deployments will meet it within a year and most will not recognize it when they do. And the reason this chapter opens the part of the book about how responsibility is shared is that silent degradation is not one failure with one owner. It is six failures at once, and each one belongs to a different person, and that is exactly why no one catches it.
Six failures, six owners, one blind spot
Silent degradation is six drift vectors closing on each other simultaneously, and the simultaneity is the whole difficulty, because any one of them, alone, is manageable by the person who owns it. The trouble is that no one owns all six, and the composite lives in the gaps between the owners.
The first vector is attentional. The human supervising the agent gets worse at supervising it precisely because it is reliable, the paradox from the opening of this book, the vigilance that erodes with success. Whoever sits in the supervisor’s seat owns this one, and they own it from inside the very degradation they are supposed to catch, which is why someone other than the supervisor has to be watching the supervisor.
The second is substrate. The ground the agent stands on moves on many axes independently. A vendor updates the model on a Saturday. The training cutoff recedes. The retrieval corpus shifts, a tool’s API evolves, the system prompt accretes small improvements each tested alone and never together, a guardrail tightens and a workflow that used to complete now pauses. Traditional software has one substrate and thirty years of regression tooling to catch it when it moves. An agentic system has eight substrates, mostly unwatched by default, and they compose into a product of drift rates that amplifies rather than cancels. This vector belongs to the architect, who knows what the system stands on, and the eval owner, who could measure when the ground shifts, if anyone asked them to run the regression. Neither owns the trigger, the silent Saturday update that no one was watching for.
The third is observational, and it is the strangest, because the thing that decays is the detection system itself. The engineer who built the monitoring rotates to another team. The product manager who knew why a particular threshold was chosen leaves the company. The dashboard from month two still runs in month eighteen and no one remembers what it was for. A monitoring layer with no institutional memory becomes decoration that everyone trusts and no one understands. This vector belongs to whoever runs the agent in production and to the institutional memory of the team, which is to say it belongs to no one in particular, because institutional memory is precisely the thing a team loses when people rotate.
The fourth is compensatory, and it is the reason a degrading agent can look healthy for a year and a half. Users learn the agent’s quirks. They prompt around its failures, they double-check the tricky cases in a second tool, they quietly build a shadow workflow that carries the load the agent has stopped carrying well. Their own metric, did I finish my work, stays green. The system’s metric, did the agent do its job, was often never measured, so from the outside the product looks successful while a parallel human workflow holds it up. The people who can see this vector are the domain experts and the users themselves, the ones doing the compensating, and they are the people least likely to be in the room where the agent’s health is reviewed.
The fifth is security, and it is specific to agentic systems and missing from the standard drift literature. The agent that was secure at launch faces attackers who learn its tool boundary, find the injection patterns that work, and share them. The agent did not get weaker. The threat environment got better at exploiting it. A launch red-team report is a snapshot of a moving target, and the patterns that failed against the agent in spring may succeed by autumn. This vector belongs to security and the red-team, who are, on most teams, not a standing function but a thing that happened once before launch.
The sixth is scope, and it is the one a product manager is most likely to find by accident. The agent ships with a defined mandate. Over time users find new uses, the team adjusts prompts for adjacent cases, an integration widens the tool boundary by half a step for a one-off that then sticks, and the agent ends up doing work nobody authorized. An agent that shipped to flag anomalies in financial postings is, six months later, also adjusting payment terms, and the answer to “who decided it should do that” is “no one decided it exactly, but here is the chain of small accommodations that put it there.” This vector belongs to the product manager, who owns the mandate, and the architect, who owns the tools that have to be revoked, and it is the only one of the six that lands cleanly on the seat the prior books would have assigned the whole problem to.
Read those six as a list of owners rather than a list of mechanisms, and the structure of the failure becomes visible: six vectors, at least five distinct owners, and not one of them can see the other five. The degradation is real and it is large, and it is invisible not because it is hidden but because it is distributed, smeared across six sightlines such that no single sightline contains it.
The six gathered into one view make the distribution impossible to miss, and impossible to fix by hiring one watcher:
| Drift vector | What is decaying | Owner |
|---|---|---|
| Attentional | The supervisor’s vigilance, eroded by the agent’s reliability | The supervisor, who is inside the decay they must catch |
| Substrate | The ground under the agent: model, corpus, APIs, prompt, guardrails | The architect and the eval owner |
| Observational | The detection system itself, as people rotate and memory fades | Production ops and the team’s institutional memory |
| Compensatory | Hidden, as users quietly build shadow workflows around the failures | The domain experts and the users doing the compensating |
| Security | The threat environment learning the agent faster than the agent hardens | Security and the red-team |
| Scope | The mandate, widening through small unauthorized accommodations | The product manager and the architect |
Five distinct owners, six vectors, and the composite that none of them can see alone is the failure.
Why this is the proof, not just an example
This is the chapter that earns the argument the rest of the book makes, so it is worth being explicit about what it proves. The previous two books in this series treated silent degradation as the product manager’s problem, and prescribed the product manager’s disciplines for it: version the instruments, ask the currency question, commission the audit. Those disciplines are right and this chapter keeps them. But the framing was an artifact of writing from one seat, and it is quietly impossible, because a product manager cannot see five of the six vectors. They cannot measure the supervisor’s eroding attention, which is a human-factors and people-management problem. They cannot detect the silent substrate update, which is the architect’s and the eval owner’s. They cannot see the shadow workflow, which lives with the domain experts. They cannot re-run the red-team, which is security’s. The product manager can watch scope, the one vector the old framing happened to put in their lap, and call the agent healthy on the strength of the one sixth of the picture they can actually see.
That is not a criticism of product managers. It is the structural fact the whole book rests on, stated in its clearest instance. Silent degradation is invisible to any single owner by construction, because it is the composition of six failures that live in six different domains. The only thing that can see it is a team that has deliberately assigned each vector an owner and built a place where the six of them compare notes, because the failure lives precisely in the gaps between them and the gaps are only visible from above. A reliable agent does not need a smarter supervisor. It needs a supervisory system, distributed across the people who can each see one face of the decay, with someone whose job is to assemble the six faces into the one composite none of them can see alone. The characteristic failure of these products is designed, by its nature, to defeat any single sightline. Silent degradation is what one-owner supervision looks like eighteen months in.
Giving each drift an owner
The disciplines survive the re-vantaging; what changes is that each one acquires a named owner instead of defaulting to the product manager who cannot perform it.
Version the instruments and re-calibrate them on the frontier-model clock, because every observation instrument an agentic system uses has a useful life of roughly eighteen months. Several frontier generations ship in that window, and each one resets the calibration of every instrument, because capability shifts and refusal boundaries move and context behavior changes. An instrument not re-calibrated since the last frontier release is measuring the agent you used to have. This is eval-owner work, scheduled and executed, with the architect ensuring the platform still emits the raw events the instruments read, and the product manager owning only the decision about what a tripped instrument should do. It is not a dashboard maintained on the side by whoever has time. It is a versioned product artifact with an owner and a release calendar.
Ask the currency question, at contract and on a cadence: when was the model’s training data last refreshed, on what corpus, with which retractions pulled, and are behavioral change notes published at each release. The buyer who asks first is the one whose agent is still doing its job at month eighteen, because absence of an answer is an answer, a risk accepted without being named. But knowing which retractions matter, that a particular pharmaceutical study was pulled, that a cancer-staging edition was revised, that an accounting standard changed on its own calendar, is domain-expert knowledge, not procurement’s. The currency question is asked by procurement and the product manager together and answered for relevance by the domain expert, and the re-ask goes on the renewal calendar owned by whoever will still be here next year. The currency of a bought model is one instance of a larger problem, the whole picture of the world the agent reasons over and whether it is sufficient and still true, and the next chapter takes up that picture and the seat that owns it.
And commission a truly external audit, on the pattern that finally caught the sepsis model: not a vendor audit, which tests the vendor’s self-understanding, and not a compliance audit, which tests conformance to a written standard, but an independent party with no commercial relationship measuring real performance on a held-out sample the vendor did not help design. The audit does not test the agent. It tests the monitoring. If the team’s distributed supervision is healthy, the audit confirms it cheaply. If it is not, the audit is how the team finds out before the story becomes the headline about the company whose agent did something in production that no one knew it had been doing for months. That list exists already. The sepsis model is on it, and it took years to land there only because an academic team had the funding and the curiosity to run one number. A company does not get that team. It gets whatever supervisory system it built, and whether that system could see all six vectors or only the one the product manager was left holding.
For one deployed agent, the six vectors are six questions with one shape: who owns this, and when did they last look. The seam audit at the back of this book runs that question across all six and across every other seam this part is about; the answer that matters here is the one that comes back empty, because a vector no one owns is the vector the agent is degrading through right now, silently, while the dashboard stays green.