Building the accountability framework for AI in financial crime: what regulators will ask

Your regulator will not ask whether your AI is accurate.

‍

They will ask three harder questions. What is this agent authorized to do on its own? What happens when it is wrong? And where is the audit trail that lets you prove either answer? If you cannot respond to those in plain language, accuracy metrics will not save you in the exam room.

‍

This is the conversation most teams are avoiding, because it is harder than the capability conversation. Deciding what agentic AI can do is a demo. Deciding what it is accountable for is governance. The good news is that AI accountability in financial crime is not new territory. It is the same discipline you already apply to human analysts, extended to a new kind of worker.

‍

The shift you have to internalize: model risk to agent risk

For a decade, model governance asked one central question. Is the model accurate? That question still matters, but it is no longer the one that decides your exposure. With agents, the question becomes: what actions is this system authorized to take, what triggers an escalation, and can it fail in ways I did not anticipate?

‍

That is a move from model risk to agent risk, and it changes what you govern. A model produces a score. An agent takes steps. You are no longer only validating an output, you are supervising a set of authorized actions. Which is exactly why knowing where your system sits on the four levels of agentic AI is the first governance decision, not a technical footnote. A Level 2 copilot and a Level 4 autonomous agent carry completely different accountability burdens, and treating them the same is how programs get caught out.

‍

The four pillars that make an agent defensible

Before you talk about guardrails, the underlying system has to be built for reliability. There are four pillars worth checking, and they are as much governance questions as engineering ones.

‍

Determinism controls. Production systems run compliance tasks at or near zero temperature, so the model is not being creative when creativity is the last thing you want. This is a baseline, not a differentiator.

‍

Deterministic code generation. The reliable pattern is an agent that writes SQL or Python to pull specific fields, rather than letting the model interpret raw data directly. The model reasons, the code executes. It sounds like a technical detail. It is actually the difference between an audit trail you can reproduce and one you cannot.

‍

Evaluation datasets. You validate against real, human-reviewed decisions with known outcomes, benchmarked against your best analysts, not average performance. Ask any vendor how many examples, what the source is, and what threshold a task clears before it ships.

‍

Context engineering. Each task gets the minimum sufficient information rather than everything available, because model quality degrades as the context window fills. Scoped context is not just about accuracy. It is about being able to explain, later, exactly what the agent looked at when it made a call.

‍

What a defensible audit trail actually contains

Here is where a lot of "explainable AI" claims fall apart. Showing what the AI decided is not an audit trail. Showing the decision, the reasoning chain behind it, the specific evidence it cited, and a confidence score, that is an audit trail. All three, on every action.

‍

This is also the connection to how you buy. One of the clearest tells of agent washing is a system that logs outcomes without the reasoning. It looks fine in a demo and collapses the first time an examiner asks why a particular alert was closed. If a vendor cannot show you the reasoning chain and the cited evidence for a specific historical decision, you do not have a defensible system, you have a black box with better lighting. Insist on AI agents whose every action is logged with its reasoning and its sources.

‍

Dual-direction QA: the agent checks the humans too

The most mature deployments I have seen do something that surprises people. The AI audits the humans. Analysts review the agent's work, and the agent reviews the analysts' decisions for consistency and adherence to standards. Quality assurance runs in both directions.

‍

That matters for accountability because humans are non-deterministic decision-makers too. You already manage that reality with QA audits, random sampling, training, and oversight. Dual-direction QA is that same framework applied evenly. It also happens to catch drift on both sides, which is the point.

‍

The EU AI Act question: production, not roadmap

When AI governance comes up, the EU AI Act is the reference point, and the question to ask is deceptively simple. Are you compliant in production today, or is compliance on the roadmap? Those are very different statements, and vendors blur them constantly. Ask whether they can produce the documentation demonstrating in-production compliance on demand. A roadmap is a promise. Production is a fact.

‍

The guardrails that turn principles into practice

Accountability is not a policy document, it is a set of controls that are live in the system. Four that should be non-negotiable:

‍

Least-privilege execution, where the agent operates with the minimum permissions each specific task requires and no more. Configurable escalation triggers that pause the agent the moment conditions cross a defined threshold, rather than letting it guess. Random sampling of agent outputs for defensibility review. And clear scope constraints that define what the agent can and cannot do, so uncertainty defaults to a safe, conservative action instead of an inventive one.

‍

None of this is exotic. It is the operational spine of a program you can defend.

‍

Where this is heading

Two things worth watching. Regulators are moving toward guidance on task-based agents supervised by overlaying manager agents for quality control, so the institutions that build real accountability frameworks now will be the ones ready when that guidance lands. And the strategic question will keep shifting away from raw capability toward authority and control. The teams that treat accountability as a feature to bolt on later will spend next year retrofitting. The ones that treat it as the foundation will spend next year scaling.

‍

The question to answer before you go live

Sit your team down and answer one thing honestly, for every agent you plan to deploy: what is it authorized to do on its own, what happens the moment it is uncertain, and can you show a regulator the reasoning behind any decision it made six months ago? If the answers are clear, you are ready to scale. If they are not, you have found your work.

‍

If you want help building that framework, talk to our team about how accountability is designed into the platform rather than added after. And for the full governance model, including the seven dimensions to evaluate any vendor on, download the Chartis report we co-produced, The Agentic AI Maturity Spectrum in Financial Crime.

‍

Frequently asked questions

‍

What is the difference between model risk and agent risk?

Model risk asks whether a model's output is accurate. Agent risk asks what actions a system is authorized to take, what triggers escalation, and how it can fail. Agents take steps, not just produce scores, so governing them means supervising authorized actions, not only validating outputs.

‍

What makes an AI decision defensible to a regulator?

A defensible decision comes with a full audit trail: the decision itself, the reasoning chain behind it, the specific evidence cited, and a confidence score. Logging only the outcome is not enough.

‍

What is dual-direction QA?

Dual-direction QA means humans review the AI's work and the AI reviews human decisions for consistency and standards adherence. It extends the QA, sampling, and oversight framework institutions already use for human analysts to cover AI agents as well.

‍

What should I ask a vendor about the EU AI Act?

Ask whether they are compliant in production today rather than on a roadmap, and whether they can produce documentation of that in-production compliance on demand. Roadmap compliance is a promise, not a control.

‍