Matthew Green on Anthropic’s New Cryptanalysis Results

Matthew Green, renowned for his ability to clarify complex cryptographic concepts, has sounded a cautious note regarding the latest advancements in large language models like those developed by Anthropic. Contrary to the notion that these models are merely "glorified autocomplete" or that progress is stagnating, Green is quick to assert that the models are demonstrating remarkable intelligence and are improving at an impressive pace.

Specific evidence of progress includes measurable improvements over the past five months in tasks Green has tasked the models with. This progression is evident across various problem sets, suggesting a rapid evolution in the models' capabilities. However, Green warns that there may not be a clear ceiling to these improvements, as the models continue to evolve rapidly.

The perception of models as "dumb" is often attributed to a narrow, limited interaction with basic AI search results, which does not reflect the full potential of more advanced tools. These tools, available for as little as $20, are not out of reach for most users. Moreover, those who believe models are already super-intelligent or that Artificial General Intelligence (AGI) has arrived are also likely to be disappointed.

Green likens the interaction with these models to swimming in a pond with a rapidly changing depth. Initially, the models provide helpful assistance, but crossing a certain threshold can result in a sudden drop in performance, leaving the user back to "swim on their own."