Programme design, metrics, and institutional integrity

Every revenue authority or FIU eventually reaches the point where the volume of reports, returns and transactional data flowing in vastly exceeds the capacity of human analysts to review case by case, and the institution must decide how to build a risk-scoring or analytics function to triage that volume. I have advised on several such build-outs across African revenue authorities and FIUs, and the recurring lesson is that the technology is rarely the binding constraint — governance, data quality and institutional incentives are.

Start with what the analytics function is actually for. In an FIU, it typically means scoring incoming suspicious transaction reports (STRs) and currency transaction reports against a combination of rule-based red flags (structuring patterns, PEP linkages, high-risk-jurisdiction counterparties, rapid in-and-out movement) and, increasingly, statistical or machine-learning models trained on historical case outcomes to predict which reports are most likely to correspond to genuine underlying criminality. In a revenue authority, it typically means scoring filed returns and third-party data (CRS files, customs declarations, VAT invoicing data) to select cases for audit, again using a mix of rule-based indicators (mismatches between declared income and asset holdings, industry-specific margin anomalies, related-party transaction volumes inconsistent with arm's-length pricing) and predictive models trained on the outcomes of past audits.

The design choice between rule-based and machine-learning approaches is not merely technical; it has real institutional-integrity consequences. Rule-based systems are transparent and auditable, an official facing a legal challenge or an oversight review can explain exactly why a case was flagged, because the rule is legible. Machine-learning models, particularly more opaque architectures, can achieve higher predictive accuracy on held-out historical data but are harder to explain to an affected taxpayer, a court, or a legislative oversight committee, and; critically, are vulnerable to reproducing and amplifying whatever bias exists in the historical case outcomes they were trained on. If past audit selection was itself biased (say, disproportionately targeting small informal-sector traders because they were easier targets, while systematically under-auditing complex multinational structures because they required scarce specialist skills the authority lacked), a model trained on that history will learn and perpetuate exactly that bias, dressed up in the appearance of statistical objectivity. I tell every analytics team I advise: a model is not neutral merely because it is mathematical: it inherits every bias present in its training data, and unexamined deployment of such models in a fragile institutional-trust environment is a serious risk, not a shortcut.

A second, distinct failure mode is overfitting to known typologiesOverfitting to known typologiesA model's weakness in detecting genuinely novel schemes because it was trained only on previously identified and caught patterns. at the expense of detecting genuinely novel schemes. Risk-scoring models are trained on cases that were caught; by construction, they are weaker at flagging the structurally different scheme that hasn't yet been identified as a typology, because there is no positive training example for the model to learn from. This is precisely why FATF guidance and virtually every serious analytics practitioner insist that automated scoring must remain a triage tool that feeds a human analytical review, not a replacement for it, and why a meaningful proportion of an FIU or revenue authority's analytical capacity should remain devoted to open-ended, hypothesis-driven investigation of patterns the models were never trained to look for.

A third failure mode, common in under-resourced institutions, is what I call "scoring without capacityScoring without capacityThe failure mode of generating more correctly flagged high-risk cases than an institution has resources to investigate, creating an accountability exposure." — building a sophisticated risk-scoring system that correctly identifies far more high-risk cases than the institution has capacity to actually investigate, resulting in a growing backlog of correctly flagged but never-actioned cases. This is worse than not scoring at all in one specific respect: it creates a documented institutional record, discoverable in any later audit, mutual evaluation or oversight review, that the authority knew about the risk and failed to act — a serious accountability exposure. The correct design discipline is to size the scoring threshold to actual investigative capacity, accepting a higher score cut-off (and therefore fewer, more confidently flagged cases) rather than generating a flood of flagged cases the institution cannot realistically work.

Data quality is the unglamorous but decisive determinant of whether any of this works. Analytics built on inconsistent taxpayer identification numbers, unreconciled beneficial-ownership data, or STR narrative fields filled in inconsistently by reporting entities will simply produce garbage scores regardless of the sophistication of the modelling technique layered on top. I have seen institutions spend disproportionate resources on machine-learning model development while the underlying registry data, the beneficial-ownership register, the taxpayer master file — remains riddled with duplicate records, stale addresses and unresolved identity-matching problems that no model can compensate for. The sequencing discipline is: fix identity resolutionIdentity resolutionThe data-quality process of reliably matching records referring to the same individual or entity across disparate datasets and registries. and data quality first, deploy transparent rule-based scoringRule-based scoringA risk-scoring approach using explicit, legible indicators (e.g. structuring patterns, PEP status) that can be individually explained and audited. second, and only then layer in more sophisticated predictive modelling once there is a sufficient, well-governed data foundation and enough historical outcome data to train against responsibly.

Finally, governance of the model itself needs the same institutional rigour as governance of any other enforcement power. Every risk-scoring model deployed in a state institution with coercive powers over citizens' financial affairs should have a documented validation methodology, periodic bias and performance audits against demographic and sectoral breakdowns (not just aggregate accuracy), a clear human-override and appeal pathway for affected persons, and an accountable owner within the institution who can explain and defend the model's design and outcomes to oversight bodies, courts, and, where BO or personal data is involved — a data-protection authority. This last point is the subject of the next lesson.

The analytics capability described in the previous lesson does not operate in a legal vacuum. It sits directly on top of personal data — beneficial-ownership information, financial-transaction records, and increasingly, biometric and behavioural data, and the legal limits on how that data can be collected, stored, matched and shared have shifted materially in the last several years, above all through the jurisprudence of the Court of Justice of the European Union (CJEU).

The decisive judgment for this field is the CJEU's ruling of 22 November 2022 in the joined cases C-37/20 and C-601/20 (WM and Sovim SA v Luxembourg Business Registers), which struck down the provision of the EU's Fifth Anti-Money Laundering Directive requiring that beneficial-ownership registers be accessible to the general public without any need to demonstrate a legitimate interest. The Court's reasoning is worth understanding precisely because it will keep shaping BO-register design well beyond the EU: the Court held that general public access to beneficial-ownership data constitutes a serious interference with the fundamental rights to respect for private life and to protection of personal data under Articles 7 and 8 of the EU Charter of Fundamental Rights, and that this interference was not proportionate to the (legitimate) objective of preventing money laundering, because unrestricted public access went further than was strictly necessary and lacked adequate safeguards against misuse of the data (including, the Court noted, use for purposes unrelated to AML, such as identifying wealthy individuals for unrelated commercial or even criminal purposes). The immediate practical effect was that several member states suspended or restricted public access to their BO registers pending redesign, and the subsequent EU AML Package; Regulation (EU) 2024/1624 (AMLR) and Directive (EU) 2024/1640 (AMLD6), was drafted with this ruling squarely in mind, reintroducing access on a "legitimate interest" basis (extending explicitly to journalists and civil-society organisations working on AML, among others) rather than unrestricted public access, an attempt to thread the needle between transparency objectives and the proportionality standard the Court set.

This matters directly for officials outside the EU, for two reasons. First, FATF's own R.24 revision (March 2022) and R.25 revision (February 2023) require countries to ensure beneficial-ownership information is adequate, accurate and available to competent authorities in a timely manner, but do not themselves mandate unrestricted public access — so jurisdictions designing or reforming BO registers now have a genuine choice, informed by the CJEU's proportionality reasoning even where the CJEU has no direct jurisdiction, about how to balance transparency against data-protection and even personal-security concerns (a live issue in jurisdictions where wealthy or politically prominent individuals face genuine kidnapping or extortion risk if beneficial-ownership data is fully public). Second, and more broadly, the proportionality methodology the Court applied — is the measure suitable to the objective, is it necessary (i.e., is there a less-restrictive alternative that achieves the same objective), and is it proportionate stricto sensu (does the severity of the interference outweigh the benefit)?, is a transferable analytical tool that any official designing a data-sharing, risk-scoring or surveillance-adjacent AML measure should apply to their own design choices, regardless of jurisdiction, because it captures a real and durable tension inherent in this field rather than a peculiarly European one.

With that legal grounding in place, the second half of this lesson turns to measuring whether an AML/CFT system actually works; a question distinct from, and in some respects more important than, whether it is merely compliant on paper. FATF's Effectiveness Methodology, used in every Mutual Evaluation since the 2013 revision of the assessment methodology, assesses each country against 11 Immediate Outcomes (IOs), covering: (IO1) risk understanding and coordination; (IO2) international cooperation; (IO3) supervision; (IO4) preventive measures by the private sector; (IO5) legal persons and arrangements (beneficial ownership); (IO6) financial intelligence use; (IO7) money-laundering investigation and prosecution; (IO8) confiscation; (IO9) terrorist-financing investigation and prosecution; (IO10) preventing terrorist-financing abuse of non-profit organisations and proliferation financing, and (IO11) proliferation-financing targeted financial sanctions. Each IO is rated High, Substantial, Moderate or Low effectiveness, a deliberately distinct scale from the parallel Technical Compliance ratings (Compliant/Largely Compliant/Partially Compliant/Non-Compliant) assessing whether the country's laws match the FATF standards on paper. The critical insight for officials designing a national programme is that Technical Compliance and Effectiveness frequently diverge sharply: a country can have excellent laws on the books (strong Technical Compliance) and still show Low effectiveness on the IOs that measure whether those laws produce real-world outcomes — genuine prosecutions, genuine confiscations, genuine use of financial intelligence to disrupt actual criminal networks. Most FSRB Mutual Evaluations of developing-country members over the past decade have shown exactly this pattern: reasonable-to-good Technical Compliance ratings following legislative reform programmes, but Moderate-to-Low Effectiveness ratings, because implementation, institutional capacity, and genuine enforcement follow-through lag well behind legislative drafting.

I use the IO framework directly with the revenue authorities and FIUs I advise as an internal self-assessment tool, independent of the formal Mutual Evaluation cycle (which typically runs only once every five to seven years per FSRB member): running an internal, honest IO6-and-IO7-focused self-assessment annually — how much financial intelligence is actually disseminated and actually used to open investigations (IO6), and how many of those investigations actually result in prosecution or conviction proportionate to the country's real ML/TF risk profile (IO7), surfaces implementation gaps far earlier than waiting for the next formal Mutual Evaluation to reveal them, and gives management a genuine internal accountability tool rather than a purely external, once-a-cycle compliance exercise.

This closing lesson deals with the institutional-integrity dimension of AML/CFT work that technical training too often skips: what happens when the enforcement system's own credibility is externally challenged through greylisting, and what happens when the enforcement function itself becomes corrupted from within, and how the individuals who detect and report that corruption are, or are not — protected.

FATF's public "increased monitoring" list (commonly called the "grey list") identifies jurisdictions with strategic AML/CFT deficiencies that have committed to an action plan to address them, distinct from the much smaller "Call for Action" (black) list reserved for the most serious cases. Greylisting has become one of the most consequential reputational and market-access instruments in this field, because correspondent banks, institutional investors and multinational counterparties treat listing as a de facto risk signal irrespective of the underlying technical detail, often triggering de-riskingDe-riskingCorrespondent banks and counterparties withdrawing or restricting relationships with entities in listed or high-risk jurisdictions, often disproportionate to the specific deficiencies identified. (correspondent banks withdrawing relationships) that imposes real economic cost well beyond what the specific deficiencies identified would independently justify. South Africa was placed on the FATF grey list in February 2023, following its 2021 Mutual Evaluation Report which identified deficiencies including in the investigation and prosecution of serious money laundering, particularly in cases linked to state capture, and in beneficial-ownership transparency and supervision of certain sectors. South Africa's action plan focused on demonstrating improved investigation and prosecution of complex money-laundering and state-capture-linked cases, strengthening beneficial-ownership data at the Companies and Intellectual Property Commission, and improving supervisory effectiveness — and South Africa was removed from the list in October 2025, in a delisting that FATF and South African authorities both attributed to demonstrated improvement across those specific action-plan items, though observers noted the two-and-a-half-year listing period had already imposed real reputational and correspondent-banking costs.

Nigeria was placed on the grey list in February 2023 alongside South Africa, with an action plan addressing beneficial-ownership information, financial intelligence use, and terrorist-financing-related supervision and investigation, reflecting deficiencies identified in its own Mutual Evaluation. Kenya was placed on the grey list in February 2024, with an action plan covering risk-based supervision of non-bank financial institutions and designated non-financial businesses and professions, beneficial-ownership transparency, and improving the investigation and prosecution of money laundering and terrorist-financing cases consistent with Kenya's risk profile. The pattern across all three cases is instructive and consistent with the Technical-Compliance-versus-Effectiveness divergence discussed in the previous lesson: each listing followed a Mutual Evaluation identifying Effectiveness gaps, real-world enforcement, beneficial-ownership data quality, and investigation/prosecution outcomes; rather than purely legislative deficiencies, confirming that greylisting today is substantially an effectiveness-and-implementation instrument, not merely a legislative-drafting scorecard.

The lesson for programme designers is direct: an action plan built around the specific Effectiveness gaps a Mutual Evaluation identified, with visible, evidenced improvement in real prosecutions, real confiscations, and real beneficial-ownership data quality (not just amended statutes), is what FATF's ICRG process and the relevant FSRB actually assess when considering delisting, and jurisdictions that treat greylisting purely as a legislative-drafting exercise, without the accompanying institutional capacity and enforcement follow-through, extend their own listing period unnecessarily.

The final and, in my view, most important theme of this entire Mastery level is the corruption of the enforcement function itself. Every framework covered across this curriculum, customer due diligence, suspicious-transaction reporting, beneficial-ownership registers, exchange of information, asset recovery, risk-scoring analytics — depends on the integrity of the officials and institutions operating it. Where the enforcement function itself is captured — where FIU staff leak STRs to targets, where prosecutors selectively decline cases involving politically connected individuals, where customs or tax officials extract bribes to suppress adverse findings, where asset-recovery proceeds are themselves diverted before reaching any legitimate use, the entire apparatus becomes not simply ineffective but actively harmful, because it retains the legitimating appearance of a functioning AML/CFT system while providing cover for exactly the conduct it purports to prevent. State-capture-era South Africa is the most thoroughly documented recent example: the Zondo Commission's findings detailed how key law-enforcement and revenue institutions were deliberately weakened and, in some documented instances, actively used to shield politically connected individuals and networks from investigation over an extended period, a central input into the Effectiveness deficiencies that later drove the FATF greylisting itself.

Protecting the people who detect and report this internal corruption is therefore not a peripheral human-resources concern; it is a structural precondition for the entire system's integrity. Whistleblower and investigator protection needs to address several distinct risks: protection from retaliatory dismissal or demotion (requiring clear statutory protection and, in the strongest models, reversal of the burden onto the employer to justify any adverse action taken against a report-linked employee); protection from physical threat, which is a real and, in several documented African and Latin American cases, lethal risk for investigators and whistleblowers exposing organised-crime-linked corruption, requiring genuine witness and investigator protection programme capacity, not merely a statutory promise; protection of anonymity and confidentiality of the reporting channel itself, including internal-affairs and external-oversight-body reporting routes independent of the chain of command the report concerns (since reporting to a compromised superior defeats the purpose entirely), and career protection, ensuring that officials who make politically difficult but professionally correct decisions are not systematically passed over for promotion or transferred to marginal postings, a subtler but pervasive form of institutional discouragement that formal whistleblower statutes rarely reach. I close this level with this point deliberately: every technical instrument in this curriculum is only as good as the willingness of an individual official, at some specific moment, to apply it honestly against pressure not to, and building institutions that protect and reward that individual choice is, in the end, the actual work of AML/CFT reform.