My GPU Database Lost to a Single CPU Thread. The Bug Was One Constant, 128x Too Small

A GPU SQL engine maintained by the reporter suffered a performance issue when running benchmarks on Apple Silicon. The GPU took 1.7 times longer than a single CPU thread processing the same workload. The bug was traced back to a single integer constant used in the slot-lock kernel, which was 128 times too small.

The issue was discovered by systematically sweeping the group cardinality parameter in the workload. The reporter found that when the group cardinality was between 256 and 1024, the GPU performance dropped significantly, taking 2.4 times longer than the CPU. This sudden slowdown was caused by a change in the algorithm from using a radix-opt path to a slot-lock path.

The slot-lock kernel builds 4096 fixed hash partitions, each with threadgroup-resident slot tables. However, when the group cardinality is as low as 1,024, all the partitions hash into the same handful of slots, causing thousands of threads to serialize behind slot locks and significantly slowing down the processing.

The reporter fixed the issue by changing the constant from 1024 to 131,072. This adjustment eliminated the performance regression and improved the GPU's speed relative to the CPU in most cases. The reporter also confirmed that this bug was not being triggered in real-world scenarios through the main application's dispatch logic, making it a rare situation where the buggy code was only exercised during the benchmark process.