Sloc Cloc and Code - Did I just get Buster Scrugged?

A recent github issue raised against the Scc tool, comparing it to another code counter, Mezura, sparked a deeper analysis. The original scc tool took longer to complete its task, processing around 86,000 files compared to Mezura's 67,000. Despite this, the performance gap wasn't justified. To investigate, the author ran the scc tool using strace, a diagnostic tool that shows system calls and signals.

The results revealed that scc was making nearly 2.6 times more kernel calls compared to Mezura, even though it was only dealing with an extra 28% more files. This discrepancy was traced back to scc's file processing pipeline, which used a single goroutine for stat file processing, leading to a buildup of work in the queue. By introducing multiple goroutines to handle the workload, and switching from os.Open() to syscall.Open(), the number of futex calls dropped significantly, resulting in a faster runtime.

This optimization also fixed some concurrency bugs. Further improvements included reducing the number of file system calls and optimizing the epoll process. These changes led to scc becoming faster than Mezura, and potentially other similar tools.