E-means: 10x faster k-means at equal loss

e-means claims to be up to 45 times faster than Lloyd's algorithm on GPUs when clustering data sets with 100,000 points into 200 groups. This speedup is achieved while maintaining equal or better clustering quality. e-means accomplishes this by using mini-batch updates with exponential moving averages, eliminating the need for hyperparameter tuning.

The algorithm works on static datasets, unlike VQ-VAE's codebook learning, where data constantly shifts. By optimizing the initialization, dead-cluster revival strategy, and cosine-scheduled decay rate, e-means avoids the trap of local minima that can trap Lloyd's algorithm. Additionally, e-means offers a finalization stage that further reduces training loss with minimal additional runtime.

Its parallelizable nature allows it to achieve substantial wall-clock speedups, especially on GPUs. The core of e-means is its centroid update rule, which replaces Lloyd's full-pass dataset averaging with an exponential moving average, allowing it to track recent assignments rather than the entire history. This enables e-means to escape local minima more effectively than Lloyd, which tends to favor established local minima.