AI failed to properly patch software flaws 74% of the time, 1Password's study warns

A new study has found that artificial intelligence failed to properly patch software flaws 74% of the time, according to research by 1Password's security team, Off-By-1 Labs. The FLAWED study explored what happens when large language models (LLMs) are given free rein to generate fixes for complex vulnerabilities in open source software.

The researchers selected six recently disclosed vulnerabilities in open source software and tested the capabilities of AI models to produce patches. They generated 6,080 patch attempts, with LLMs successfully creating suitable patches only 26% of the time. In 21% of results, the patches fixed the bug but also altered the application's behavior. The AI models failed 53.9% of the time, either by adding new bugs or both failing to create a patch and introducing new vulnerabilities.

The main issue appears to be that regardless of environmental conditions or prompts, LLMs generated "Fix-Like Artifacts with Embedded Defects," which superficially appear to fix the vulnerability but lack robust security mechanisms and may introduce additional bugs. These "FLAWED" patches can change an application's typical behavior.

Off-By-1 Labs released its tooling, FLAWED, on GitHub for researchers to conduct their own studies. The researchers suggest that while LLMs excel at discovering vulnerabilities, they are currently only effective at patching a narrow subset of them. They emphasize the need for human oversight in the patch process, as having an informed understanding of which bugs are most impactful in a codebase is crucial for human defenders and AI tooling.