Re-balancing Deflate Compression Levels

After optimizing the Go deflate compression, the author discovered that benchmark results displayed an issue: a significant jump between level 1 and 2, and minimal gains above level 5. Level 4 was identified as the optimal setting, with a noticeable speed drop when moving to level 5 (24% slower, 0.7% better compression). The default compression level was deemed excessively costly, while levels 6 to 8 lacked justification for their presence.

The author aimed to rebalance the compression levels to better reflect trade-offs, focusing on a reasonable speed for web servers and general use. They tested the changes using different datasets - 'enwik9' (a Wikipedia XML dump), Matt Mahoney's 10GB corpus (a wide variety of files), and a homemade 'Webserver test suite' (548 typical webserver files).

Through multiple iterations, the author determined level 6 as the optimal default, offering a balance between speed and compression. Level 5 became the recommended setting, providing a middle ground with a 4-5x speed increase and only a 2-5% compression loss. The author encourages testing the improvements in the provided PR branch, with the expectation of merging them into Go 1.7.