From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts
Test-time scaling produces cohorts of reasoning rollouts, yet there is no standard label-free account of how their internal computation reorganizes as inference unfolds. We introduce routing effective rank deff, the entropy-effective dimensionality of a cross-rollout graph built from MoE expert-routing similarity. Across ten MoE configurations and five math/science benchmarks, deff exhibits a…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.