Kronecker Delta Rule for Long Context Linear Attention
We propose KronLSTM, a linear attention architecture based on the Kronecker Delta Rule, a novel update rule for associative matrix memories. KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associations while modifying the memory only along a single direction in matrix space. This minimally invasive memory revision reduces interference with previously…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.