AI Needs Access to Data. That Doesn't Mean It Needs to See It | Opinion

AI systems require access to data for their operation, but this does not imply they need unrestricted visibility into that data. When a bank utilizes customer records to enhance services or uncover fraudulent activities, the institution bears the responsibility for safeguarding that information. This accountability should encompass the moment the data is being processed.

While organizations are expanding their use of AI, robust privacy commitments necessitate more than just written policies; they require technical enforcement at the data level. Leaders should inquire whether their systems effectively restrict access to sensitive records, rather than merely instructing personnel to exercise caution.

To illustrate this point, consider an AI assistant assisting an employee in resolving a payment query. If this AI can access unrelated account details, even a straightforward inquiry could inadvertently disclose information beyond its intended purpose. This scenario emphasizes the importance of constructing boundaries within the system, complete with approval requirements for sensitive actions and a clear demarcation of when access should cease.

Employees should be able to operate within these predefined limits without needing to compensate for design oversights that grant the assistant unnecessary permissions. The scrutiny must extend to the processing phase as well. While encryption at rest secures stored records and encryption in transit safeguards data during transmission, neither measure alone ensures the security of information when software interacts with it.

A protected database may still enable a process that renders sensitive details readable. Historically, conventional processing methods typically necessitated decrypting encrypted information before analysis. However, ongoing research into encryption in use is unveiling alternatives that enable computation on encrypted data without exposing the underlying records to the party performing the calculation.

Although it is not feasible for every application to achieve this level of security at an acceptable speed or cost, executives should investigate the feasibility of such solutions before assuming exposure is an unavoidable consequence. This investigation is integral to responsible stewardship, alongside controls governing who can request analysis and receive its outcomes.

A financial-services experiment conducted in September 2025 by SWIFT demonstrates the potential of protected collaboration. Involving 13 banks and generating 10 million artificial transactions, the experiment combined privacy-enhancing technologies with federated learning, a method that trains models locally at each institution rather than aggregating customer records into a single dataset.

According to SWIFT, this collaborative model was twice as effective as a model trained on a single institution’s dataset in identifying known fraudulent transactions. While this experiment utilized synthetic data and does not definitively prove equivalent results in live banking operations, it does demonstrate that encryption in use can contribute to improved analysis.

For me, the key takeaway is that privacy protection should be viewed as a facilitator of analysis. Institutions should explore whether safeguards can support appropriate collaboration before presuming their only options are unrestricted sharing or abandoning valuable work. Regulatory standards also acknowledge the necessity of protecting data during processing.

Article 6 of the regulatory technical standards supporting the EU’s DORA framework addresses encryption of data in use when deemed necessary, contingent on data classification and risk assessment. In cases where encryption is not feasible, protected processing environments or equivalent measures are required. I interpret this conditional provision as a rationale to thoroughly examine processing risks.

While technology cannot resolve every concern, organizations must still evaluate whether information should be collected, whether they possess the appropriate authorization to utilize it, including when the data has been de-identified. Computing demands, integration, access permissions, and the treatment of results must also be considered.

Leaders can allocate these protective measures within existing budgets. Postponing such measures may inadvertently transfer avoidable risks to customers. Conversely, dismissing every sensitive-data application without evaluating the available safeguards might also entail missing opportunities to enhance fraud detection or other beneficial services.

Before endorsing such a project, executives should mandate a comprehensive review outlining what information becomes accessible, who can access it, and which protective measures are appropriate for the task. This review should incorporate testing, delineate the authority for using the information, designate an accountable executive, and provide an explanation of the residual risks.

It is important to acknowledge that no approach can eradicate all potential for failure. However, organizations should be prepared to justify any exposure instead of passively accepting it as an unalterable condition. Responsible AI should integrate the protection of information into its utilization. Jackie Peters is the co-founder and CEO of Blind Insight, a technology company that creates encrypted-data search and analytics tools for organizations managing sensitive information.