EROFS Disables LZ4 Rolling Decompression Due To Data Corruption Possibility
The EROFS read-only file-system popular for embedded systems, containers, and other use-cases has resorted to temporarily disabling its LZ4 rolling decompression support due to data corruption concerns...
AWS engineers have discovered a potential issue with LZ4 rolling decompression, which could lead to corrupted data in specific datasets. The root cause appears to be uncontrolled backward memory copies within the current LZ4 implementation, which breaks the assumptions of the rolling decompression optimization. Since the kernel LZ4 codebase is not under their control and requires more time to address, AWS recommends disabling LZ4 rolling decompression temporarily to ensure data correctness in real production environments.
LZ4 rolling decompression was introduced to reduce memory usage, but it works by referring to the most recent 64 KiB of decompressed data, requiring a bounded rolling window of temporary pages. This optimization may need to be re-evaluated once the official LZ4 code ensures forward memory copies for non-overlapping ranges or when EROFS maintains its own LZ4 implementation.
Written by urgent.news from Phoronix's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
