Earning Each Permission Before Taking the Next One

The gap between a pilot and production is where most of this industry's money has been spent, and the reason is almost never the technology. This bank had lived through both ways these programmes die.

• The first death is procurement. The pilot succeeds and the demo lands. Then the vendor is asked where conversation data physically resides, what the agent can reach on the network, and whether session isolation is a property of the code or of the machine. The answers disappoint, the second line declines, and nobody records it as a failure. It simply stops.
• The second death is moving too fast. A bank impatient after the first failure goes live broadly against a containment number of 70% to 90% sold from a deck. Those numbers are real for mature flows in low-risk verticals. Banking intents are not those flows.
• One bad transcript ends it. Pushed to hit a number it was never going to reach safely, the agent answers something it should have escalated. It reaches a regulator, a journalist, or an executive committee, the programme is pulled, and the institution does not try again for two years.
• The quiet third problem: no measured baseline. When the previous pilot ended, nobody could say whether it had worked. Containment had improved, handle time was better, and the decision went to whoever argued most confidently in the room.
• A programme that cannot prove its result cannot survive a new sponsor. Which is what the requirement actually was: not a better model, but trust earned in increments too small for any one of them to kill the programme.

CreateOS deployed in four phases, and the sequencing was the product. We ran it in the order the risk function asked for rather than the order that would have produced the fastest headline.

• Measure before touching anything. The bank's actual cost per contact, containment, handle time, and satisfaction, not an industry benchmark. That baseline went into the contract as the yardstick, so the result could not be argued away by a future sponsor and we could not be held to a number nobody had established.
• Integration is the real work, and it takes weeks. Core systems, knowledge base, authentication, and the voice, chat, and messaging channels along allowlisted paths, self-hosted inside the bank's boundary, with grounded content loaded for a deliberately narrow set of intents.
• Live, but read-only, for three weeks. Real customer traffic on balances, FAQs, product questions, and card status, with full logging and human oversight, and no ability to move money.
• That phase exists to generate evidence, not value. Most vendors skip it, and skipping it is the most common cause of the second death. The risk function watched a real record accumulate instead of reading a promise in a document.
• Actions enabled one at a time, never in a batch. Each gated by the kernel egress allowlist and its own confirmation control, with irreversible actions requiring customer confirmation and, where the bank chose, a named human.
• Containment earned, not forced to a number on a slide. We committed to a lift into the 50% to 60% range on the right intent mix and said plainly we would not promise 90%. We would rather lose the deal at the proposal than lose it at the transcript.

The program went live in nine weeks and then expanded, which is the outcome the bank had failed to reach twice before.

Measured against the Phase 0 baseline the bank itself established:

• Containment lifted from 30% to 60%, deflecting 1.5 million contacts a year from the human queue
• Average handle time on assisted contacts down 20%
• Customer satisfaction maintained and improved from a 79% baseline
• $7.5 million to $10 million a year in cost reduction on a $25 million base, a 30% to 40% cut
• Zero unauthorized actions, enforced in the kernel
• Zero cross-customer data leakage, enforced by per-session isolation
• 100% audit coverage of every interaction and every action

• Every number has a measured before, agreed in writing. A programme that can prove its result survives a change of sponsor, a change of CFO, and a regulator's question. The previous pilot could not, and it did not.
• No bad transcript, and the read-only phase is why. Grounded accuracy and session isolation were demonstrated on production traffic for three weeks while the agent was structurally incapable of moving money. There was no window in which survival depended on the agent not making a mistake it was capable of making.
• Containment is still rising, because the evidence allows it. Each intent added is one the safety record now supports. The bank is expanding the risk band because it can, not because a contract obliges it.

The four phases