How Does mHC Use Its Residual Streams? Selective Routing and Near-Identity Mixing
Hyper-Connections and their manifold-constrained variant mHC widen a residual pathway from one stream to n, yet how trained models use this capacity remains unclear: how broadly blocks read and write, how strongly the residual pathway mixes streams, and whether the streams carry distinct representations. We examine these properties in the four-stream residual pathway of DeepSeek-V4-Flash using…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
