The demo looked great. The Artificial Intelligence (AI) answered questions confidently, handled the expected scenarios smoothly, and impressed everyone in the room. Now imagine putting that same agent in front of 10,000 real customers.
That’s where things get interesting — and not always in a good way.
Financial services firms are racing to deploy AI agents across customer support, lending, fraud detection, claims processing and back-office operations. The appeal is real: faster responses, lower costs, and the ability to process information at a scale no human team could match. But somewhere between “impressive prototype” and “production system”, a gap opens up — and that gap is where most of the risk lives.
Here’s the thing about demos: they’re curated. The questions are clear, the data is clean, and the right answer is already known. Real customers don’t work like that. They give you half the information. They ask questions your workflow never anticipated. They challenge policies, describe situations that don’t fit any category, and occasionally push the system somewhere it was never designed to go.
In most industries, a stumble at that point is a minor inconvenience. In financial services, it can become a compliance failure, a reputational crisis, or a direct harm to a vulnerable customer.
The model is not the whole story
This is the part that catches organisations off guard. They invest in a capable AI model, run tests, see it perform well — and assume the hard part is done. But the real risk rarely lives in the model itself. It lives in the surrounding system: what data the AI can access, what decisions it’s allowed to influence, where its authority ends, and whether anyone has actually tested those limits.
A well-trained model inside a poorly-designed workflow will still produce unsafe outcomes.
Consider a common example. An AI agent handles routine account queries without a problem. Then a customer mentions they’re in financial hardship. Or they dispute a payment. Or they ask for something the system technically cannot authorise. What happens next? Does the agent escalate? Does it give a confident but wrong answer? Does it even recognise it’s out of its depth?
These aren’t hypothetical edge cases. They’re the kind of situations that happen constantly — and they’re exactly the situations a standard demo won’t show you.
Testing for failure, not just success
Juan Carlos Melgar (pictured), Founder of OpsTwin Finance, puts it directly, “Most organisations start by asking whether the AI works. A better question is whether the AI is ready to operate inside a real financial environment, where the information is imperfect, the risks are higher and the consequences are more serious.”
That reframe matters. Functionality testing tells you what an AI does when everything goes right. What you also need to know is how it behaves when things go wrong — when policies conflict, when a customer pushes past the intended scope, when the information is incomplete, or when the system is being asked to make a judgment it shouldn’t make.
Catching those failure modes before production is the entire point. And it’s exactly what OpsTwin Finance was built to do.
The startup helps financial teams evaluate AI-enabled workflows through synthetic financial scenarios — realistic, pressure-tested situations that probe the boundaries of an agent’s behaviour without requiring real customer data or live system integration. The output is structured risk analysis and concrete evidence, the kind of documentation that compliance, legal and risk teams can actually use.
Readiness isn’t a checkbox
One of the more useful things Melgar says is that AI readiness isn’t an event — it’s a practice. “It’s not a box that should be checked once. It’s an ongoing process of testing assumptions, reviewing controls and understanding where human oversight is still necessary.”
That means the questions organisations should be asking aren’t one-time questions. What is the agent authorised to do? What must it never do? At what point does a human need to take over? How does the system behave when information is missing — and how does the organisation prove that those guardrails actually work?
These questions sound simple. They’re often the difference between a prototype that performs well in a controlled setting and a system that can be trusted with real customers, real money and sensitive financial data.
The organisations that get the most out of AI in financial services probably won’t be the ones that move fastest. They’ll be the ones that understood both sides of the equation — what the technology can do, and where it can fail — before they went to production.
As Melgar puts it, “Trust in AI should not come from how impressive the demo looks. It should come from evidence of how the system behaves when things become difficult, that is the principle behind OpsTwin Finance: Test before trust.”