OpenAI and Meta warn U.S. judicial panel about deepfake evidence

OpenAI and Meta Platforms have warned a U.S. federal judicial panel that AI-generated deepfakes are becoming so sophisticated that authenticating audio and video evidence in court will become increasingly challenging. The representatives shared their concerns at the Advisory Committee on Evidence Rules, which has been working on tackling the risks of AI-generated evidence for the past three years.

Shira Anderson, policy counsel at OpenAI, told the panel that AI images are becoming increasingly realistic, and she doubts a future scientific method will be able to detect them. The Federal Judiciary Center's survey from March revealed that only 15 out of 931 judges had encountered cases where evidence was challenged due to being a deepfake.

US District Judge Jesse Furman, who chairs the panel, highlighted that while academic predictions suggest courts will face deepfakes, it hasn't been a common issue in actual cases. Devin Norder, Meta's director of product management, stated that AI-generated media cannot be differentiated from authentic content by humans, and creating such media requires minimal cost and time.

Companies like OpenAI and Meta have developed tools to verify visual content, but new laws in California and the European Union mandate watermarking to detect deepfakes. However, Anderson warned that each company uses its own system, and there is no universal method to reliably detect all watermarks or verify visual content's authenticity.

She emphasized that while these systems will continue to improve, they will never be completely reliable.