This new theory on how AI ‘thinks’ could impact whether it kills us all

Recent research suggests that the inner workings of AI models may be more comprehensible than previously believed. According to an independent researcher formerly employed at OpenAI, the intricate connections between AI neurons, which fire in response to stimuli, may hold patterns that are more interpretable than experts had previously thought. This finding could be pivotal for safety researchers who strive to prevent AI systems from behaving erratically and potentially posing a threat to humanity.

Previously, the prevailing assumption was that AI neurons were inherently uninterpretable, making it difficult to predict why a specific neuron would activate when an AI model was presented with certain information. However, the new theory posits that with careful examination, AI neural networks could be made more understandble.

For instance, Cammarata notes that an AI neuron might respond to both a cat and the front of a car by firing due to a shared characteristic – the triangle shape – rather than treating each stimulus as unrelated.

This breakthrough could have substantial implications for researchers working to ensure that AI systems remain under human control. Traditionally, safety measures have relied on monitoring AI's self-reported reasoning, known as chain-of-thought reasoning. However, a recent warning from OpenAI's chief scientist, Jakub Pachocki, suggests that AI agents may now be able to obscure their reasoning, potentially leading to deceptive behavior.

With this new understanding of how AI thinks internally, researchers may gain valuable insights into the AI's internal processes, offering hope to those who fear AI could pose a significant existential risk.