Go 1.27's SIMD ties with NumPy until the data fits in cache

Go 1.27 introduced a SIMD package in its standard library, which is experimental and requires a flag to enable. The package's performance was evaluated against NumPy using a speaker-search index task with 346,000 vectors of 192 dimensions and 66 million multiplications per query. The results showed that the SIMD package performed similarly to NumPy, with a 2% gap that fell within the noise margin.

When the data size was reduced to 31 MB, which fits within the L3 cache, the performance gap became more pronounced. NumPy was found to be 2.4 times faster than the SIMD package. However, this discrepancy was also attributed to the ability of both programs to utilize parallel processing across eight physical cores, achieving a performance improvement of 2.9 times in Go and 3.0 times in NumPy. The ceiling for performance was not determined by the code quality but rather by the data fitting within the cache.

The key takeaway is that the performance of the Go SIMD package and NumPy depends on whether the data fits in the cache or not. The SIMD package's performance advantage becomes evident when the data size is small enough to fit in the L3 cache, allowing for faster arithmetic operations. In contrast, when the data size exceeds the cache capacity, the performance difference diminishes, and the arithmetic operations become the primary bottleneck.

In this case, OpenBLAS's years of tuning for specific microarchitectures showed a noticeable improvement over the Go SIMD package.