Why AI agents fail in month three

The pattern is consistent enough to have a shape. Month one: the pilot handles the sample cases well and the steering committee is delighted. Month two: volume goes up and the team starts noticing overrides. Month three: a case goes wrong in a way that reaches a customer or a regulator, confidence collapses, and the agent is quietly moved back to "assisted mode" where it stays.

The instinct is to blame the model. Almost every time we look, the model was fine. What was missing was context: the agent had been given a task and a set of documents, and had never been told how this organisation actually decides.

Capability is not the constraint

Modern models can read a bank statement, classify a document, draft a letter and reason through a policy. What they cannot do is know that in your firm, self-employed applicants need six months of statements rather than the three the product matrix says, because the matrix is out of date and the underwriters quietly corrected it two years ago.

They cannot know that a screenshot of a payslip is never acceptable, that documents in another language always go to a person, or that the third re-request to a broker is escalated rather than sent. Those are not capabilities. They are context. And in most enterprises that context is scattered, contradictory and partly unwritten.

The three failures of month three

1. The unknown rule. The agent meets a case the pilot sample never contained, applies the documented rule, and gets it wrong because the documented rule was superseded in practice.
2. The contradiction. Two sources disagree. The agent picks one. Nobody had decided which was right, so the agent's choice is arbitrary and, worse, inconsistent from case to case.
3. The missing stop. The agent should have handed the case to a person and did not, because nobody defined the conditions under which it must stop.

None of these are fixed by a better model. All of them are fixed by better context.

Building the layer that is missing

The work is unglamorous and decisive. Inventory every source of rules for the process: policy, procedure, product matrices, team spreadsheets, precedent cases, and the senior people who carry the rest in their heads. Put the sources side by side and find where they disagree. Then sit with the decision owners and resolve each contradiction, recording who decided and why.

Structure the result so an engineer can load it and a risk manager can audit it: the rules, the order of precedence when rules conflict, the exception playbooks, the boundaries, the stop conditions. Version it. Give each part an owner. Test the agent against it before it sees a live case.

Month three still arrives. With the context layer in place it arrives as a set of overrides that feed back into the pack, not as a loss of confidence. That is what separates the agents that reach production from the ones that stay in the pilot.