Person Hides Prompt Injection in Legal Filing Telling AI to Side With Them

Matthew Elliott, representing himself in a Connecticut court, concealed instructions in his legal filing to manipulate artificial intelligence (AI). These instructions, known as "prompt injections," directed an AI system to side with Elliott and ensure its output aligned with the filing. The hidden commands were written in small, 3-point white font, making them nearly invisible to the human eye.

Elliott filed the lawsuit against the New York Bariatric Group in October, alleging privacy violations and discrimination. In a filing in July, he included lengthy notes intended for AI systems, instructing the hypothetical AI model to accurately reflect the filing and to ensure remediation. The court noticed unusual white space in Elliott's pleadings and identified the concealed text, which was formatted to be barely visible to humans but fully legible to software.

The court found that the hidden instructions were directed towards AI systems reviewing the filing, aiming to produce favorable output to Elliott. Judge Walter Spader Jr. emphasized that the attempt, even if unserious, raises serious concerns about the integrity of the legal system. He noted that while AI tools hold promise, especially for those who cannot afford a lawyer, using them dishonestly is improper.

Spader Jr. called for serious reflection on the use of AI in legal proceedings, warning that such practices may become more common.