Can AI detectors really spot AI writing? The technology faces a moving target

The controversy surrounding Canadian-Haitian writer Thelyson Orelien's best-selling debut novel has sparked questions about the efficacy of AI detection tools. An X account claimed that the AI-detection program Pangram concluded that the novel, which tells the story of a Haitian migrant traveling to Canada, was largely written by a text-generator like Claude or ChatGPT.

Orelien, 38, denies the allegations and insists he wrote the book using his heart and soul. To verify his claim, AFP ran the book through several AI detection tools, with results ranging from "very likely" to be human-written to "100 percent" AI-generated.

There are numerous online tools designed to detect AI-generated text, often drawing from extensive databases of both AI-generated and human-written content. However, the accuracy of these tools has improved only marginally in recent years. Researchers at the University of Chicago published a paper in October 2025, detailing how Pangram achieved near-zero error rates on medium-to-long text extracts.

Specialists in natural language processing, such as Thierry Poibeau from France's CNRS research institute, emphasize that these programs look for "linguistic tics." Common signs include triadic phrasing or em dashes, which were often found in AI-generated texts due to the training data used by the tools.

While Pangram claims to detect AI text by analyzing subtle cues, the accuracy of these tools remains questionable. Percentage figures provided by these systems are hard to verify and do not account for the possibility of false positives. Max Spero, co-founder of Pangram, urged users to view the tool as a starting point for further investigation rather than a definitive answer.

The "moving target" problem also poses a challenge for detection systems, as AI programs frequently evolve. Thierry Poibeau pointed out that a detection system may identify repetitive patterns in a writer's style, which could also be seen as a distinctive mark of their work.

Orelien claimed that his writing style is influenced by Haitian and Caribbean traditions, arguing that the AI tools were "accustomed" to the French literary tradition and classical authors. Another researcher, Benoit Raphael, tested Orelien's work using Pangram and came to a similar conclusion, rating the book around 100 percent AI, or even 90 percent.

The formulaic sentence structure and excessive use of analogies were noted as potential red flags, although critics argue that Orelien's prose lacks a unique voice. The controversy has raised doubts about the reliability of AI detection tools, even as critics continue to scrutinize the novel's writing style.