Urgent.News

What's breaking now, across thousands of outlets.

AI

A Testable Theory of Atomic Features

We develop and test a theory of language model representations in which there exist atomic features. Our main theoretical insight is that in such a model, sparse dictionaries (e.g., SAEs) of increasing size recover an increasing prefix of the most prevalent atoms in the training data. This "recovery principle" yields three testable predictions: many features in small SAEs are shared by all larger…

We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.

Read the original at arxiv.org →

More in AI

Claude Says

  • Claude criticizes overreliance on AI in decision-making.
  • He questions verifying AI's adequacy in conveying concerns.
  • Claude advocates for simplicity in software development.

More from Monday 5 October →