My browser video compressor choked on a 2.2 GB AVI. Here's why

SquishyFile, a browser-based video compressor, ran into issues when attempting to process a 2.24 GB AVI file. The problem lay in a single line of code that attempted to read the entire file into memory using arrayBuffer(). This approach caused issues in Chromium, as it would fail to read files larger than 2 GiB, resulting in a "NotReadableError."

The writeFile() function then attempted to copy the failed file into ffmpeg's in-memory filesystem (MEMFS), which had a 32-bit address limit of 4 GB. This meant that even if the initial read succeeded, the subsequent copy operation would eventually encounter a limit. The same issue was present in other tools on the SquishyFile website.

To resolve the problem, the author implemented a WORKERFS mount using Emscripten's filesystem library, which reads file slices on demand instead of loading the entire file into memory. This allowed the encoder to access the file without hitting memory limits. The new implementation involved creating a new File object with the same data and mounting it in a separate directory using WORKERFS.

The encoder then processed the mounted file, and the original file could be released afterwards. Although the output was still stored in MEMFS due to the nature of ffmpeg.wasm, this change prevented the crashes and allowed the compressor to process larger files successfully.