Was I wrong about open models & cyber risks?

In recent weeks, a significant shift in the cybersecurity landscape has become increasingly apparent. Coining the term "Black October" after a surge of critical vulnerability disclosures, including PHP2Shell and GitLab, the cyber risks associated with AI models have started to materialize. The news cycle seemed to stabilize briefly as I began writing this article, but then another major crypto security incident at Binance occurred, highlighting the ongoing challenges.

The cybersecurity space has seen a rapid acceleration in vulnerability disclosures, with experts strained to patch the internet. This accelerating trend is likely to continue, leading to an exponential increase in vulnerabilities. One key revelation from this story is that the AI safety community has been overly concerned about the hypothetical risks posed by open-weight AI models, such as Claude Mythos.

While the prospect of AI models causing substantial harm is not entirely dismissed, the evidence suggests that the impact would be more of an acceleration than a step change in risk.

Prominent figures in the AI safety community have expressed exaggerated concerns about the potential cyber weapon capabilities of open-weight models, with statements ranging from catastrophic societal destabilization to a cornucopia of zero-day exploits. However, based on the available evidence, these predictions seem to be incorrect. The actual exploit found in Firefox, for example, demonstrates the multi-step process required to turn an exploit into a harmful attack.

The cybersecurity landscape is currently experiencing a wave of critical vulnerabilities being discovered in popular tools, with fixes being built to mitigate the risks. This ongoing process of vulnerability discovery and patching is crucial in maintaining the security of our digital infrastructure.