Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it. The result keeps full attention in only 6 of its 18 blocks. The other 12 use short convolutions whose memory is two timesteps wide no matter how long the…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
