Why tiny JPEGs look different in Chrome

A seemingly rendering issue was actually a result of a clever JPEG decoding optimization in Chrome. The problem arose when a colleague's computer displayed a logo differently than the reporter's. The logo appeared thinner and more true to the original image on their screen. To test, the reporter created a new image and found that Chrome rendered it thicker when scaled down to small sizes. To understand why, the reporter dove into the technical details of how JPEG images are rendered.

The reporter explained that rendering a small image from a JPEG involves fully decompressing it in memory before scaling it down. This creates a large memory footprint, especially for high-resolution images. However, most of the information in the large image is lost when it is scaled down. The high-frequency details, such as fine textures and patterns, are discarded during scaling.

Interestingly, the information lost during scaling is not random. When an image is heavily scaled down, the high-frequency details that change rapidly from pixel to pixel are the first to disappear. For example, the fine details of a tree's leaves and bark become a solid green blob and a brown trunk when scaled down. The high-frequency information is mixed together, so some of it still survives in the scaled-down version.

The reporter then delved into the technical aspects of JPEG compression. When an image is compressed, it is divided into 8 × 8 blocks, which are then converted into the frequency domain using a DCT (Discrete Cosine Transform). The lowest frequency in each block represents a flat color, while the highest frequency resembles a checkerboard pattern. These frequency components, or basis functions, are used to represent the image in a more efficient manner.

During JPEG compression, the lossy compression occurs when the coefficients representing these frequency components are stored. The lower-frequency information, which represents the overall structure of the image, is preserved, while the high-frequency details are discarded to some extent.

To address the issue of rendering small JPEGs, Chrome employs a clever optimization. Instead of fully decompressing the entire JPEG and then scaling it down, Chrome uses a technique called partial IDCT scaling. This allows Chrome to skip the coefficients for the high-frequency parts of the image and use only the necessary ones for the coarse version of the image. By decoding only the lower-frequency data, Chrome can render the image at a smaller scale without fully expanding the original image first.

This partial IDCT scaling approach provides a faster and more memory-efficient way to render small JPEGs. The decoded image takes less space and is faster to uncompress since it skips a significant portion of the coefficients. Chrome's Skia rendering engine, which handles image decoding and rendering, delegates this task to libjpeg-turbo, a library that implements partial IDCT scaling.

However, the reporter cautioned that this optimization is more suitable for photographs rather than icons or other small images. JPEG's design and optimizations are tailored to the human perception of photographs, where the trade-off between image quality and file size is acceptable. For icons, vector formats like SVG are preferable to maintain sharpness and detail at small sizes.