Benchmarking Wild vs Mold

Wild recently faced a setback in its linker benchmarks after being updated with Mold. These benchmarks indicate Wild lagging behind Mold, contrary to its previous performance. This analysis aims to uncover the cause of this disparity. The most recent Wild benchmarks were conducted on a single machine, with a key configuration difference being the presence of an output file from a prior linker run.

Mold's benchmarks, however, delete this file between linker invocations, potentially affecting performance. This impact varies based on the filesystem used. Wild's past benchmarks were run on tmpfs, a choice likely misguided since most users don't build on tmpfs. Mold, on the other hand, uses ext4, a more fitting choice. Moving forward, a mix of both filesystems will likely be utilized.

Another factor is that Mold's benchmarks pass --no-fork, reducing shutdown costs, whereas Wild leaves this setting at default only when measuring memory consumption. To achieve benchmarks similar to Mold's, Wild has been configured to use ext4, delete the file between runs, and pass --no-fork. Here is a subset of Mold's benchmark results for comparison: When considering the Wild/Mold ratios in a table format, the results are as close as expected, given the differences in CPU architecture, cache sizes, RAM, etc. By replicating Mold's configuration (ext4+delete+no-fork), we can better understand the differences in benchmark results.

The clang-release benchmark, one that Wild has published, was analyzed. Configurations used in Mold's benchmarks (ext4+delete+no-fork) and Wild's historical approach (tmpfs+no-delete+fork) were tested. For the remainder of the post, a tmpfs+no-delete+fork configuration will be used. Wild's performance peaks when allowed to fork and with an existing output file on a tmpfs, unlike the Mold's configuration.

However, this is mainly due to Wild lacking OS-specific optimizations for efficient file creation and writing on non-tmpfs filesystems. Mold's author details these optimizations in the paper "mold: A Massively Parallel Linker." These, along with hugepages to map the file, have been incorporated into Wild for the next release. Despite this, a significant performance gap remains between Wild's August 4th benchmarks and Mold's August 28th benchmarks.

To investigate further, Wild and Mold have been benchmarked over the past year for the clang-release benchmark. The results show that Mold has substantially improved. Wild's August 4th benchmarks were conducted before Mold's 2.42.0 and 2.42.1 releases, which introduced major gains. Earlier in this post, efforts were made to replicate Mold's benchmarks on an M1 Mac.

Replicating the Threadripper results, however, remains a challenge. My 16 core Ryzen 9955hx with 92GiB RAM is not a low-end machine, but it doesn't compare to a 64 core Threadripper with 384GiB of RAM. The substantial difference in performance, beyond the previously discussed factors, could be attributed to Wild's usage of 128 threads while Mold runs with 32.

On a 16 core (32 thread) machine, Wild's performance continues to improve with increased threads (see graph below). This guesswork stems from the lack of a Threadripper for testing. If anyone with access to a Threadripper is interested in benchmarking Wild with varying thread counts, please reach out.