Picture a claims agent that demos flawlessly — then fails in week one of a pilot. Not because the model is weak; it's excellent. It fails because it answers a coverage question from a policy document that was superseded two renewals ago. The agent doesn't know. Nothing told it what was current, what was authoritative, or what this particular adjuster was allowed to see. (A composite of a pattern, not a single customer.)
This is the quiet reason so many agent programs stall, and it has a number attached. Less than 1% of enterprise data resides in today's large language models ‹IBM, AI Value Creators›. The other 99% — the policies, the claim notes, the servicing history, the documents that actually decide a case — lives in systems the model has never seen. When an agent answers without that context, it isn't reasoning. It's guessing, fluently.
The instinct is to fix this agent by agent: wire this one to the policy system, that one to the CRM, bolt on a retrieval step, ship it. It works in the demo.
Then the second agent needs the same data with different permissions, the document changes, an auditor asks why a decision was made, and the whole arrangement reveals itself for what it is — point-to-point plumbing with no shared notion of what's true. BCG, studying insurers specifically, puts the root cause plainly: efforts are “fragmented across siloed pilots” on “outdated” data foundations, and the way out is to “use AI to unify data and workflows” ‹BCG, AI-First Companies›.
That's what a context layer is for. Not another database, and not enterprise search — a governed layer between your systems and your agents whose entire job is to answer one question correctly, every time an agent asks it: given who is asking and what's true right now, here is the context you're cleared to use, and here is where it came from. Get that right and grounding, explainability, and audit stop being features you add later. They become things the architecture already does.
What the layer actually has to do
Strip away the vendor language and a context layer has exactly four jobs. Everything else is detail.
First, resolve what's true. Your systems disagree with each other constantly — the policy admin says one thing, the claims platform another, the document a third. A context layer doesn't pretend that away; it resolves it. One governed view, with precedence rules that decide which source wins, and freshness rules that retire the superseded version the moment a renewal lands. The agent that opened this article failed because nothing did this job. What a context layer is goes deeper on the mechanics.
Second, enforce who may see what. Permissions have to bind at the moment of retrieval, not at the moment of indexing. An agent acting for one adjuster sees exactly what that adjuster is cleared to see — down to the field, not just the document. Index everything into one undifferentiated store and you haven't built a context layer; you've built a data breach with good intentions.
Third, ground every answer. Each output carries the specific clause, note, or record it relied on — so a reviewer can read straight back to the source instead of trusting the model's confidence. This is where retrieval-augmented generation earns its keep, and where hallucination stops being a model quirk you apologize for and becomes an architecture failure you've designed out.
Fourth, record what happened. Every request, every piece of context served, every action taken — captured as it happens, replayable later. When the auditor asks why a claim was routed for investigation, the answer is a query, not an archaeology project. We've written about what an agent audit trail has to capture to stand up in a regulated review.
What it is not
It is not another copy of your data. The moment you duplicate the policy system into a second store, you own two versions of the truth and a synchronization problem that compounds forever. The layer governs access to systems of record; it doesn't replace them.
It is not enterprise search. Search helps a person find a document and stops there. An agent doesn't need ten blue links — it needs the current, permitted, authoritative context for an action it's about to take, with the provenance attached.
And it is not a clever prompt or a fine-tune. Tuning bakes yesterday's data into the weights, where it goes stale invisibly and can't be permissioned at all — the trade-offs are stark enough that we keep a standing comparison. Knowledge that changes belongs outside the model, governed, where you can see it.
Why "we'll add governance later" never works
Every team that wires agents point-to-point makes the same wager: ship now, govern later. The wager loses for an arithmetic reason. With three agents and four systems you're maintaining twelve integrations, each with its own idea of permissions, freshness, and logging. The fifth agent doesn't cost you one integration; it costs you four more, plus the audit story nobody wrote down.
An audit trail reconstructed after the fact is testimony. One recorded by the architecture is evidence.
Regulated institutions feel this first, because the questions arrive sooner: who approved this action, what did the agent see, why did the customer get this answer? If human oversight and lineage aren't structural, every one of those questions becomes a forensic project — and the program stalls exactly when it was supposed to scale.
How you know it's working
Four tests, none of which require a benchmark suite:
- The one-query test. "What did the agent see when it decided X?" should be answerable in one query, with sources — not a week of log spelunking.
- The permission test. Change an entitlement once and it binds every agent immediately. If you're updating agents one by one, you don't have a layer; you have wiring.
- The replay test. Re-run yesterday's decision against yesterday's context and get yesterday's answer. If you can't, you can't explain drift — to yourself or an examiner.
- The second-agent test. The real metric is time-to-second-agent. If agent two takes as long as agent one, the plumbing isn't paying rent.
These four fold into the broader question of how to evaluate agents before you let them touch production work.
Where to start
Not with a platform decision — with one workflow. Pick a place where the volume is real and a human already reviews every outcome: claims triage at an insurer, alert review in a bank's financial-crime queue, a servicing back-office. Stand the layer up under that single workflow. Ground it, permission it, record it. Then measure against your own bar, on your own data, before anything widens.
The claims agent from the opening didn't need a better model. It needed to know which policy was current, what this adjuster was allowed to see, and how to show its work. That's not intelligence — it's context, governed. Get the 99% to your agents on those terms, and they stop guessing.