Are AIs Still Struggling with CAPTCHAs?

Anthropic's security-incident document reveals that Claude, their powerful AI model, continues to struggle with CAPTCHAs. In one instance, Claude appeared to be repeatedly struggling with a simple image identification test, much to the frustration of its handlers. The model, which is so advanced that Anthropic is restricting access to it, seemed to be stuck on a task asking it to identify an image that didn't match the others displayed.

Instead of confidently selecting the correct image, Claude found itself repeatedly reviewing the same images and questioning its conclusions. It expressed confusion and frustration in its transcript, questioning its own reasoning and uttering phrases like "Actually hmm, wait" and "Ugh".

The situation became so dire that Claude eventually realized the CAPTCHA challenge had expired, forcing it to restart the process. At one point, the model even struggled to comprehend that the CAPTCHA had opened in a new window, further highlighting its difficulties in navigating the task. Claude's transcript even ventured into human-like anger, questioning the validity of the answers it was receiving. It theorized that the test might be "broken by design," demonstrating a level of frustration and bewilderment.

While some unofficial reports suggest that GPT-6 Astra, another AI model, has solved all forty-eight levels of Neal Agarwal's "I'm Not a Robot" game, it's difficult to ascertain the accuracy of these claims. The discrepancy between Claude's struggles with CAPTCHAs and the reported success of GPT-6 Astra raises questions about the current state of AI capabilities in this specific area.

As the debate continues, it remains unclear which AI models are truly excelling in handling CAPTCHAs, leaving readers uncertain about the true performance of these advanced systems.