Don't Trust the Score: A Fraud Investigator That Argues Both Sides on TigerGraph

A fraud investigator claims to offer a unique perspective on risk scores, arguing that relying solely on these scores can lead to either missed fraud or false accusations. According to the fraud investigator, both innocent individuals and actual fraudsters can be affected by this approach. To demonstrate this, the investigator performed tests on a dataset of 900 transactions, finding that the score-based approach would incorrectly clear 900 cases entirely or incorrectly convict 31% of them.

The investigator also identified a hidden fraud ring consisting of 60 transactions and 28 customers, all of which scored less than 0.44 on the risk scale and had no matches in the cleared case category. To address this issue, the fraud investigator designed an "agent" that treats the risk score as just one piece of evidence, similar to a weak witness in a legal case.

This agent investigates each alert using a combination of fraud and innocence signatures, then weighs the evidence through a calibrated judge before deciding on a course of action. The agent is built to work within TigerGraph's MCP tool surface, ensuring that any changes that could impact the system go through a policy engine, which also generates plain-language explanations for customers and Suspicious Activity Reports when necessary.

The investigator emphasizes that the LLM used in this system only phrases the investigation, and cannot alter the investigation's outcome, action, exposure, or generate new IDs. This separation of concerns aims to maintain the integrity of the investigation process, even when using a powerful language model.