Unity Mipmaps Beyond Smaller Textures: Temporal Stability, Streaming, and Semantic Mips

Mipmaps are typically perceived as smaller versions of a texture, but this oversimplifies their purpose and impact on rendering. A more accurate understanding treats a mip chain as a combination of three functions: a tool for stabilizing images over time, a mechanism to manage memory and bandwidth allocation, and a hierarchical system that can maintain various levels of meaning.

To illustrate the concept, consider a 4096 x 4096 texture viewed from a shallow angle on a floor. Due to the small size of a single screen pixel relative to the texture's footprint, even a minor camera movement leads to different subsets of texels being sampled, causing crawling detail, moire patterns, and flickering. Mipmaps address this issue by selecting a texel density that aligns better with the projected footprint of the object.

However, distance is not the sole determining factor. Other elements, such as projected size, UV scale, surface angle, texture resolution, projection method, bias, and anisotropy, also influence the outcome. The GPU estimates the rate of change in UV coordinates across neighboring fragments, represented by rho = max(length(ddx(texelPosition)), length(ddy(texelPosition))). The level of detail (lod) is then determined by taking the logarithm of rho.

Blending methods differ based on the filtering technique employed. Bilinear filtering blends texels within a single mip, trilinear filtering blends between two adjacent mips, and anisotropic filtering handles elongated footprints on oblique surfaces. Trilinear filtering should not be mistaken for anisotropic filtering, as they serve different purposes.

When budgeting resources, it's crucial to distinguish between the ideal Level of Detail (lod) desired for a particular scenario and the highest-resolution mip level actually available in GPU memory. For instance, a square texture's complete chain consists of 12 levels, with each successive level containing one quarter of the texels of the previous one. Consequently, the first level accounts for approximately 75% of the chain, while the remaining levels reduce to about one sixteenth of the texels.

The decision to enable or disable mipmaps depends on the specific context and the characteristics of the textures being used. Base color or ordinary emissive color, which typically requires sRGB color space, should generally have mipmaps enabled. In contrast, normal maps, which usually benefit from a linear color space, may have mipmaps disabled. Other factors, such as the presence of repeating patterns, thin text, or alpha-tested shaders, should also be considered when deciding on mipmap settings.