RipGrep musl binaries occasionally segfault during very-large searches
Article URL: https://github.com/BurntSushi/ripgrep/issues/3494 Comments URL: https://news.ycombinator.com/item?id=49133889 Points: 248 # Comments: 167
Ripgrep, the fast file searching utility, has been observed to occasionally crash during the execution of very large searches on systems using the musl C library. This issue was discovered while using the pre-built version of ripgrep bundled with OpenAI Codex, which is functionally identical to the one available at the specified GitHub release.
To recreate the problem, a large tree of files was generated using a Python script specifically designed to mimic the data patterns of the repository where the bug was first encountered. This tree contained approximately 20 gigabytes of data spread across 1.8 million individual files. Once the tree was created, a loop was initiated in the root directory to run ripgrep in an endless search for a non-existent string within the tree.
On a 24-core system with sufficient memory to accommodate the search tree in the kernel's cache, it typically took around a minute for a segmentation fault (SIGSEGV) error to occur. The crash consistently occurs at a specific line of code, which involves an integrity check on heap metadata in MUSL's mallocng library during a `calloc` operation.
The full backtrace, which documents the sequence of function calls leading up to the crash, has been provided for reference. A core dump file and the binary responsible for the crash are also linked for further investigation.
Written by urgent.news from Hacker News Best's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
