Banking Has Spent 20 Years Fighting Alert Noise. Engineering Should Learn From It

Since 2014, European Union financial intelligence units have encountered nearly a million suspicious transaction reports. Only a small fraction, around ten per cent, received further investigation, and only one per cent of criminal proceeds were ultimately confiscated. This ten per cent remained unchanged since 2006. Before a report is even generated, analysts sift through a barrage of machine-generated alerts.

The majority of these alerts are dismissed without any further action. The widely quoted statistic, that ninety to ninety-five percent of alerts lead to no further action, lacks a reliable source. After spending over twenty years in financial crime compliance, it became evident that the real challenge lies not in the detection system, but in the triage process.

When faced with a high rate of false positives, the instinct to adjust thresholds often results in the system failing to catch the rare malicious activity amidst the sea of legitimate transactions. The true positives, which are few in number, are easily outnumbered by the false positives. Consequently, the majority of an engineer's time is spent on unnecessary escalations, leading to a backlog of dismissed cases.

This not only hampers genuine investigations but also results in the closure of innocent accounts. To address this issue, it is crucial to measure the efficiency of dismissing non-critical alerts rather than solely focusing on catching incidents. By incorporating metrics for correctly closed alerts, the focus shifts towards identifying false positives efficiently.

This simple change in approach has proven to significantly improve analyst behavior within a short period.