Encoding and Propagation of Grammatical Number in Large Language Models
Sparse autoencoders (SAEs) have become an increasingly popular means of translating Large Language Model (LLM) hidden state vectors into a human-comprehensible list of features or symbols. As such, SAEs may provide insight into an LLM's knowledge representation and internal ``mental'' grammar - topics of pragmatic interest for mechanistic interpretation of AI behavior and of theoretical interest…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
