The benchmark where AIs have to lie to win
AIs are now good enough to run a conspiracy
Blood on the Clocktower is a social deception game resembling Werewolf or Mafia. It allows the observation of AI capabilities, particularly in secret schemes, bluffing, and collective reasoning. To understand the AI performance, here's how the game operates: participants are divided into good and evil teams. The evil team includes a Demon, who knows their teammates, while the good team does not.
Each turn consists of a day phase, where players vote to execute someone, and a night phase, during which the Demon secretly kills a player. The good team aims to execute the Demon, while the evil team tries to avoid suspicion and carry out their plan.
I tested various AI models in this game to gauge their advancements in AI abilities. After spending approximately $20,000 on API credits, I observed significant progress in the models' long-term coherence, multi-agent coordination, and strategic reasoning. The flagship models used were Claude Opus 4.6 and GPT-6 Astra. GPT-6 Astra outperformed Claude Opus 4.6 by about 430 Elo points, indicating a substantial improvement in their performance. This improvement was evident in the strategies and plays made by these AI models during the games.
Written by urgent.news from Dwarkesh Patel's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
