
A sponsor bank's fintech partner has run AI agents in assisted mode for six months. Now it wants the AI to auto-close 40% of transaction monitoring alerts, and the exam letter lands in 90 days. What has to be true, and what has to be in the file, before anyone signs off?
That was the case Jason Mikula, publisher of Fintech Business Weekly, put to four experts at The AI Files, Unit21's virtual summit on AI in fraud and AML:
The short answer: there's no AI-specific rulebook yet, examiners aren't waiting for one, and the BSA officer stays accountable no matter who built the model.
In April, the Federal Reserve, the OCC and the FDIC replaced SR 11-7 with SR 26-2, the new model risk management guidance. A footnote says generative and agentic AI models are novel and rapidly evolving, so they sit outside its scope. Guidance specific to those systems was expected over the summer and hasn't arrived.
So what are institutions doing in the meantime? Kevin said many are watching each other: there's a herd mentality, and nobody wants to be outside the pack. The more useful work is cross-functional. Banks are pulling model risk management together with IT and operational risk to govern AI as a shared framework rather than waiting for a dedicated one.
And no, examiners won't go easy. Kirsten acknowledged that supervisory attention has pivoted toward financial risk, but BSA is still drawing enforcement actions, some with very large numbers. Banks need sound risk management over how AI is used in BSA, guidance or not.
Kirsten, playing the examiner, said the standard request list hasn't changed much even when examiners know AI is in use. Expect the usual items:
One AI-specific item she would look for even at a bank that says it doesn't use AI: training. By late 2026, she wants to see that the BSA team's professional development covers AI. If it doesn't, that's troubling on its own.
What gets validated? Kevin split it in two. Most of what banks call AI is still traditional machine learning, and that gets validated the traditional way: does it perform as expected, is it documented, what triggers escalation or revalidation. Generative and agentic AI are harder, which is why the agencies scoped them out. The questions are the same, but the tools are less mature: is it doing what we think it should, is it hallucinating, how much does it drift, and how is that drift controlled.
Marc added the timing problem. Most banks are in a race "to be first to be third," and many default to Copilot because it feels safe. But exams look backward, often 12 to 18 months, and he expects regulation to change within three to seven years. Waiting means getting squeezed from both directions. His answer is control: understand how a model makes a decision, document it, and be able to audit it. Kirsten added that globally active institutions should look at what stricter regulators abroad are asking for and prepare those materials ahead of time.
Michael was asked what he would need to sign off on auto-closing 40% of alerts. His answer: Bakkt keeps a human in the loop on every decision, and will for some time. AI assists investigations. An analyst still reviews the evidence, decides on escalation and documents the rationale.
Bakkt is also using AI upstream, to develop and tune rules. Michael said Unit21's agents have spotted gaps in how rules trigger alerts, based on the traffic between them, and the team has turned those into new rules. The goal, he said, isn't to close more alerts. It's better quality, consistency and efficiency without weakening controls. Before he would let AI auto-close anything, he wants to see around 99% disposition agreement on very light-touch alerts.
Repeatability is the bar. Kirsten said the question before auto-closing is whether decisions are consistent and repeatable: if the same facts come up again, does the system reach the same decision? Humans aren't perfectly consistent either. Six people zig and one zags. But the bank has to be able to explain its framework clearly to examiners. Kevin put numbers on it: 99% repeatable is great, 50% is bad. Kirsten would look for something in the 97 to 99% range, adjusted for risk. A one-off, immaterial transaction is very different from large, frequent wires to another country.
Flip the scenario. Rather than starting with a fintech partner's AI auto-closing alerts, Kirsten suggested the bank use AI to test what the partner is doing. Marc pointed out that this moves oversight from sampling to reviewing 100% of the work at speed and scale. That is a much easier first step to defend in an exam. For how one fintech built exactly this kind of QA layer, see how Rippling and Zepz prove quality every day.
The catch is access. Kevin said many vendors resist giving institutions that level of visibility, and business owners eager for a new tool often sign contracts before the MSA and SOW are detailed enough to require it. Marc questioned whether MSAs keep up with the technology at all. Kevin's answer: they don't.
Many AI tools in this space sit on top of other providers. A bank works with a fintech, the fintech uses a vendor, and the vendor runs on one or more foundation models. How do you know when something in that chain changes?
Marc called it the core problem with foundation models. When a model version changes, it reasons differently, you often aren't notified, and you can lose the ability to show how a past decision was made. He also warned that general models are built for the average. An answer averaged across the smallest and largest banks won't fit a $3 billion bank in Minnesota that lends to soybean farmers. The unlock, in his view, is pairing deep human expertise with the technology.
Kevin's fix is contractual. Banks need MSAs that require vendors to disclose changes, ideally before they happen. Too often, he said, business owners drive these deals and risk teams don't have an equal seat at the table.
Michael starts vendor reviews with one question: what exactly is the AI doing, and what decisions can it influence? From there he looks at how the provider validates the technology, protects customer data and monitors performance. Fourth and fifth parties may sit outside the contract, but that doesn't remove the institution's responsibility for the risk.
Kirsten closed the section with the point that ties it together. Risk officers exist because the buck stops with them. A BSA officer is appointed by the board for exactly that reason. Using AI, a partner or a vendor doesn't shift that accountability. So the BSA officer needs a seat at the table, a clear view of how decisions are made, and support from IT, operations and vendor management. For a deeper look at this question, read building the accountability framework for AI in financial crime.
Jason closed with a lightning round.
Michael's test is a good place to start your own AI evidence file. For what that file should contain, see what 1M live AI alerts tell us about how fincrime teams adopt AI and why an AI compliance exam has almost nothing to do with your AI. If you're weighing auto-close, read how to safely auto-close alerts with AI.
Does SR 26-2 cover generative AI and AI agents?
No. SR 26-2, the model risk management guidance the Federal Reserve, OCC and FDIC issued on April 17, 2026, places generative and agentic AI outside its scope in a footnote. Banks are still expected to govern those systems through their broader risk management practices.
Will examiners go easy on AI used in BSA/AML?
No. Even with supervisory focus shifting toward financial risk, BSA enforcement continues and examiners still expect sound risk management over how AI is used in BSA/AML programs.
What do examiners need to see before AI auto-closes alerts?
Evidence that decisions are consistent and repeatable, so the same facts produce the same outcome. Panelists pointed to agreement rates around 97 to 99 percent on low-risk alerts, human review, and the ability to reconstruct any single decision end to end.
Who is accountable when a vendor's AI makes a compliance decision?
The BSA officer. Using a partner, vendor or foundation model does not shift accountability, which is why institutions need visibility into how decisions are made and contract terms that require notice of model changes.
Bakkt is licensed by the NYDFS as a Money Transmitter and for Virtual Currency Business Activity. Panelists' views are their own. This session was part of The AI Files, Unit21's virtual summit on how fraud and AML teams are putting AI to work.

Gal Perelman is the Product Marketing Lead at Unit21, where she spearheads go-to-market strategies for AI-driven risk and compliance solutions. With over a decade of experience in the fintech and fraud sectors, she has led high-impact launches for products like Watchlist Screening and AI Rule Recommendations.
Previously, Gal held marketing leadership roles at Design Pickle, Sightfull, and Lusha. She holds a Master’s degree from American University and a Bachelor’s from UCLA, and is dedicated to helping banks and fintechs navigate complex regulatory landscapes through innovative technology.