AI misuse in policing is a warning for any organization handling sensitive evidence

An investigation into alleged misuse of artificial intelligence (AI) at Derbyshire Constabulary has highlighted a broader concern that extends beyond the force. The inquiry raises the question of what happens when advanced technology is introduced into any organization where sensitive information is utilized to inform decisions that may later require explanation and justification.

Rather than excluding AI from investigations, intelligence work, or high-stakes tasks, it is crucial to deploy it responsibly, within secure, auditable systems with clear boundaries and human oversight. This approach allows AI tools to assist organizations in connecting information, managing increasing workloads, and recognizing important patterns while ensuring that every insight can be traced, explained, and defended.

The issue arises when employees turn to general-purpose tools that were not designed to meet evidential and security standards. Factors such as growing caseloads, dispersed data across teams of investigators, analysts, HR professionals, safeguarding leads, legal experts, security personnel, and senior decision-makers, make AI appealing to handle large volumes of sensitive data.

However, the problem emerges when such tools are unapproved, outside the security perimeter, lacking any record of queries or outputs. A more serious risk stems from AI's attempt to streamline case reports, potentially omitting or overlooking vital information essential for investigations. In situations where lives are at stake, these oversights are intolerable.

The risk extends to evidential integrity, regulatory compliance, and the administration of justice. To ensure safe AI usage, organizations must establish clear policies outlining sanctioned tools, their permissible applications, and the irreplaceable human judgment required. Without well-defined boundaries, employees may circumvent these guidelines under pressure.

Generalist AI tools, unsuitable for evidential purposes, can pose security, compliance, and data sovereignty issues. When sensitive intelligence is fed into third-party platforms, questions regarding security, compliance, and data retention arise. What's more, generative systems still carry a risk of error, as demonstrated by hallucination rates as high as 4.2%, even though they are becoming more reliable.

AI-generated outputs cannot be regarded as fact; they require rechecking and validation. Incorrect or unverifiable information can lead to weakened prosecutions, delayed safeguarding decisions, and opportunities remaining for those being targeted. While cautioning against the use of improperly designed AI, it is essential to recognize the role of purpose-built investigative AI.

These platforms excel in accelerating time-consuming tasks by connecting information without adding judgment. They can swiftly identify connections across various reports, records, and correspondence, presenting them for further assessment. However, human decision-making remains paramount, as these tools operate within secure environments and maintain a documented trail of queries, outputs, and decisions, facilitating accountability.

Organizations must establish governance frameworks to ensure responsible AI use, define validation processes, and clarify the roles AI should and should not play. Some activities, particularly those impacting legal proceedings, safeguarding, or individual rights, are unsuitable for automated assistance. Drawing the appropriate line is as important as choosing the right platform. The pursuit of innovation remains valuable when its outcomes can be defended.