Neural Networks Are Already Clean

Recent studies suggest that neural networks may be more straightforward and comprehensible than previously thought. Researchers now believe that a network's inherent mechanisms serve as its clear building blocks, rather than being a tangled knot of arbitrary concepts. This shift in perspective has led to optimism about the prospects for mechanistic interpretability, the field of study which seeks to understand the inner workings of neural networks.

The once-cited examples of polysemanticity and superposition, which seemed to indicate that neural networks could pack arbitrary concepts into neurons, have been reevaluated. These phenomena were mostly attributed to architectural issues rather than inherent learning principles. As a result, researchers are now focusing on studying networks directly, by examining their neurons and circuits, observing data flow through them, and performing experiments at scale to test explanations.

One of the main findings is that superposition was the incorrect explanation for the InceptionV1 model, an image classification network. Instead, architectural problems caused the later layers to overflow, and weak interpretability tools amplified these problems, making the late layers appear incoherent. However, recent improvements in network architecture have inadvertently made the network more interpretable, with 99-100% of neurons in the last layer being easily interpretable.

The initial confusion surrounding polysemanticity and superposition was also due to a misinterpretation of the data. The circuit introduced as evidence for superposition actually demonstrated ordinary generalization and composition, rather than a hidden car feature spreading into unrelated neurons. Modern language models are also more interpretable than previously believed, with preliminary audits suggesting that roughly 90% of neurons have one main job, with only a small percentage remaining unexplained.

This shift in understanding signifies a significant improvement in our ability to study and comprehend neural networks.