SIMD and Go

SIMD stands for Single Instruction, Multiple Data. This means that a single CPU instruction can perform the same operation on multiple data points simultaneously. Out of the box, the CPU does not utilize this capability when written in Go. However, with the experimental simd/archsimd and simd packages introduced in Go 1.26 and 1.27 respectively, developers can now write Go code that automatically compiles into SIMD instructions.

This article begins by explaining the fundamentals of SIMD, before delving into Go's two SIMD packages. Next, it walks through common SIMD operations with Go code samples, elaborates on how the compiler executes these operations, and finally puts everything together in a practical example.

To better understand the hardware behind SIMD, let's consider a simple task: adding two []float32 slices element by element. In standard Go, the CPU handles this by performing one addition operation per step. However, SIMD allows the CPU to perform eight additions at once. This is achieved through the use of hardware called SIMD, which has the ability to process multiple pairs of numbers concurrently.

Normally, CPUs have registers that can store a single 64-bit value. However, SIMD introduces wider vector registers, ranging from 128 to 512 bits, depending on the CPU's architecture. In this article, we will focus on 256-bit registers, which can hold eight float32 values.

Vector registers are composed of lanes, which are slots of different sizes, such as 32 lanes of 8-bit integers or 4 lanes of float64. In most of our examples, we will use 8 lanes of float32. SIMD registers are divided into these lanes, and the CPU determines how to split the data within them. Most of our examples use 8 lanes of float32.

The core operations of SIMD follow a consistent cycle: loading data into a register, performing operations on the data, and then storing the results. Let's break down each step:

The first step is loading data into a register, which is done using a vector load. This step takes eight float32 values from our input slice and stores them in the eight lanes of the register. For example, the first element of the slice goes into lane 0, the second element into lane 1, and so on. The remaining elements will be loaded in subsequent steps.

Once the data is loaded into the registers, the CPU can perform operations on them. SIMD instructions operate on a single lane at a time, performing a vertical operation. Each lane acts like a separate calculator, with no interaction between lanes. The result of each operation is stored in another register.

Finally, the results need to be stored back into memory. This can be done using a vector store, which writes all eight lanes of the register to their corresponding indices in the output slice. Alternatively, individual lanes can be extracted and stored as separate variables.

Now that we understand the SIMD cycle and how it works at the hardware level, let's see how we can implement it in Go. There are two packages available for SIMD in Go, both of which are experimental and only available when building with GOEXPERIMENT=simd.

The first package is simd/archsimd, which provides low-level access to SIMD registers and lanes. Each SIMD type corresponds to a specific register and its lane configuration. For instance, Float32x8 represents a register with eight float32 lanes (256 bits), Float32x4 has four lanes (128 bits), and Float32x16 has sixteen lanes (512 bits). The usage of these types follows the same cycle we discussed earlier: loading data, performing operations, and storing the results.

Here's an example of adding two slices using SIMD in Go, handling eight floats at a time:

```go

var a = []float32{1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0}

var b = []float32{9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0}

var c = simd.Float32x8{}

c.Load(a)

c.Add(b)

c.Store()

```

In this example, we create a new Float32x8 register and load the elements from slice `a`. Then, we add the corresponding elements from slice `b` using the Add method, and finally, we store the results in slice `c`.