OPINIÃO. A rastreabilidade nas decisões da AI

Friedrich Nietzsche's 1878 work, "Human, All Too Human," introduces the concept of the "free spirit," an individual who thinks independently and questions what is considered certain. Today, we live in an era where humans attempt to mechanize processes, especially with the increasing use of artificial intelligence (AI), often overlooking its biases and errors.

The drive for technological innovation and the desire not to fall behind have led various economic sectors, including finance, to adopt AI, altering their processes. This brings a sense of efficiency but raises concerns about the impact on confidence in decisions made by financial institutions and their clients, given that AI is created by humans and controlled by large companies with proprietary interests.

In Brazil, according to the 2026 Febraban Technology Banking Survey, 80% of institutions use AI to analyze data and enhance product and service offerings. Globally, a KPMG study shows that over three-quarters of organizations employ AI in financial planning, report preparation, and commercial analysis. Even major international banks have developed AI tools for wealth management to support their bankers and financial advisors, as well as customers, with recommendations, analyses, and summaries.

AI systems capture maximum information and encode it for machines to understand, akin to a machine-to-machine language. This shift in relationship transforms the human understanding of the message, as it is now dependent on mechanisms of compilation and processing that receive input, analyze it, and produce an output. The risk increases when considering that these AI tools are not autonomous entities, but rather belong to companies whose primary goal is to maximize market value.

There is no neutrality in AI, as they take away the neutral absorption of information, increasing the risk of losing control over the interpretation and decision-making process. Therefore, we must remain vigilant about the use of this technology across all sectors, particularly in finance. Traditionally, decisions were made based on a set of criteria that allowed for evaluating different problem elements and reaching a decision, providing a logical framework for decision-making.

This enabled subsequent comparison of results with original assessments, identifying errors, understanding what went wrong, and revising decision-making criteria. However, with AI, these intermediate steps become opaque or inaccessible. Financial service users must develop the necessary repertory to counteract this, keeping track of the step-by-step process used, obtaining information, and ultimately being able to reverse the process to perceive where errors occurred and prevent potential legal or financial damage.

The issue lies not in the technology itself but in the possibility of losing the ability to reconstruct and critically examine the path leading to a particular result, even with the use of these tools. It is crucial to follow government movements around the world to regulate AI, which advances much slower than technological progress.

In Brazil, the AI Legal Framework, proposed by Senator Rodrigo Pacheco in 2023 and impacting banks, fintechs, and insurers by classifying systems used in economically impactful decisions as "high risk," was only evaluated by the Senate in December 2024 and awaits a vote in the Chamber of Deputies. In the meantime, users, companies, and developers must assume responsibility and assess the risks of each decision made based on AI recommendations.

The problem is not the technology itself, but the risk of losing the capacity to reconstruct and critically examine the path leading to a particular outcome. Preserving the analysis of all stages of the decision-making process is essential to understand, question, and correct it if necessary. Despite the advancement of AI and the pursuit of efficiency, we must not forget the critical analytical capacity to avoid becoming the tool's victims. The tool is useful, but not innocent. Pierre Moreau is an attorney.