Speeding up GPU copies by permuting iteration spaces, or, optimising the identity function

In the world of computing, the process of copying data from one memory location to another is a common task. Compilers often generate code that performs numerous small copies, which can be inefficient. However, array languages typically deal with a relatively small number of large operations, presenting an opportunity for optimization.

Futhark, a functional array programming language, now employs a technique that potentially speeds up GPU copies by rearranging iteration spaces. The authors refer to this as "optimising the identity function" and explain how it benefits programs dealing with large operations on smaller arrays.

The fundamental concept behind this optimization is the representation of array layouts using linear memory access descriptors (LMADs). LMADs map the n-dimensional index space of an array to the flat index space of computer memory using a structured representation consisting of n pairs of integers per dimension. These pairs denote the size and stride of each dimension.

By examining the LMADs, Futhark can determine if an array is contiguous, meaning there are no gaps in memory between elements. Contiguous arrays can be copied more efficiently by using a single GPU copy function that takes advantage of hardware-level features like vectorisation. The authors mention that detecting contiguous arrays through run-time analysis is feasible, and this process is limited by the rank of the array, which is typically constant in Futhark programs.

Another optimization opportunity arises when dealing with transpositions. Transpositions can be detected by collapsing contiguous dimensions until only two dimensions remain. In some cases, implementing transpositions in a way that exploits memory hierarchy can result in near-identical performance to a full-speed memory copy.

In conclusion, Futhark's approach to accelerating GPU copies by permuting iteration spaces demonstrates the potential benefits of optimizing array copies using LMADs. While the impact may be small for some programs, this technique highlights the importance of exploiting memory layouts and making runtime analysis to improve performance.