Rust Portable SIMD Now Runs on the GPU and It Changes Everything About Cross-Platform Parallelism

For decades, GPU programming involved two distinct approaches: writing CUDA kernels in C++ or navigating OpenCL. Both required a fundamentally different programming paradigm than CPU programming. However, VectorWare has revolutionized this field by enabling Rust's portable SIMD (core::simd) to run natively on GPUs. This development represents the first major step towards bridging the gap between CPU and GPU programming that has persisted for twenty years.

Modern processors offer two levels of parallelism: thread-level parallelism, which most developers are familiar with, and SIMD (Single Instruction, Multiple Data) level parallelism, which is more challenging. SIMD allows a single instruction to operate on multiple data elements simultaneously, significantly improving the performance of numerical computations on CPUs.

On GPUs, the analogous concept is called SIMT (Single Instruction, Multiple Thread), where a group of threads, known as a warp, executes the same instruction on different data elements concurrently. Traditionally, writing SIMD code meant choosing a specific CPU architecture, such as AVX for x86 or NEON for ARM. Rust's portable SIMD (core::simd) simplified this process on the CPU side, allowing developers to write code once and have the compiler translate it into the appropriate vector instructions for the target architecture. However, this capability had not been extended to GPUs until now.

VectorWare recognized an elegant solution: a GPU warp functions as a wide vector unit, similar to SIMD. A Simd i16, 32 can map directly onto a 32-lane warp, enabling a single warp instruction to add two such vectors simultaneously. This means Rust SIMD code, which worked seamlessly on x86 CPUs with AVX, can now also run on NVIDIA GPUs without any modifications.

The abstraction layers align perfectly, allowing developers to write Rust SIMD code once and have it run efficiently on both CPUs and GPUs. This eliminates the need for separate codebases for CPU and GPU, streamlining development and enhancing portability.

The implications of this breakthrough are far-reaching. For developers, GPU programming becomes significantly more accessible, as they no longer need to learn CUDA or OpenCL. They can write Rust SIMD code, a familiar paradigm, and have it run on GPUs seamlessly. This reduces the learning curve from mastering a new programming model to simply learning a new data type.

For portability, code written against Rust's core::simd can now target x86, ARM, and GPU architectures with a single codebase, a feat previously unattainable. For Rust itself, this achievement validates the language's approach to portable abstractions. By offering portable SIMD, Rust provides memory safety across multiple platforms, a capability unmatched by other languages.

This not only enhances Rust's appeal for developers but also underscores Rust's potential as a cross-platform solution for parallel computing tasks.

Performance-wise, GPUs offer massive parallelism that remains underutilized due to the complexity of the programming model. Portable SIMD on GPUs removes this barrier, enabling Rust programs using SIMD for numerical workloads to harness GPU acceleration with minimal effort. The technical insight behind this development is that SIMT, the GPU's parallel execution model, is essentially SIMD in disguise.

While SIMT allows for thread divergence, where each thread can execute different instructions, when divergence is absent, a warp behaves like a SIMD vector. Rust's Simd T, N allows developers to write code that operates on lanes within a warp, ensuring the code stays in the non-divergent path for optimal performance. VectorWare's implementation seamlessly maps Rust's Simd T, N onto GPU warps, with the warp width matching the Simd vector width, enabling direct compilation of Rust SIMD operations to GPU warp instructions.

This direct mapping bypasses the need for emulation layers, providing a true SIMD-like experience on GPUs.

The immediate applications of this technology are widespread, particularly in numerical computing, image processing, signal processing, and simulations. Any Rust code leveraging portable SIMD for CPU acceleration can now be adapted for GPU execution with minimal changes. More significantly, the broader impact lies in the ecosystem.

Libraries, machine learning frameworks, and game engines built on top of Rust's core::simd gain GPU support automatically, unlocking a new level of performance and versatility. This integration could herald a future where the CPU/GPU distinction is less of an obstacle and more of a strategic choice, allowing developers to write code once and let the compiler determine the optimal execution platform.

However, it is important to note that this development is still in its early stages. VectorWare, a startup pioneering GPU-native software, is actively promoting this capability, and their GPU runtime is required for it to function. Performance benchmarks are currently limited, and optimizing code for GPU warp-level programming requires a deep understanding of GPU memory hierarchies and execution models.

Despite these challenges, the proof of concept demonstrated by VectorWare is a significant milestone in the programming language community's quest for seamless parallel computing across CPU and GPU architectures. This development not only advances Rust's capabilities but also paves the way for a more unified approach to high-performance computing, where developers are no longer constrained by the limitations of traditional GPU programming paradigms.