A Screenshot MAE Is Not a Material Accuracy Score

The source material discusses the distinction between screenshot difference metrics and material accuracy scores in the context of AI-assisted rendering. The author provides a specific example using PNG image files, demonstrating that a 6.407536 RGB mean absolute difference (MAE) over a full image represents a different metric than a 18.313377 MAE over a shared non-background mask.

The source also points out that the choice of averaging domain significantly impacts the reported score, and that identical background pixels do not necessarily indicate a superior renderer. The text highlights the importance of specifying averaging domains when comparing differently framed screenshots, as the same pixel differences can yield different results depending on whether the full frame or a masked region is considered.

Additionally, the source clarifies the difference between encoded RGB and decoded display output statistics, emphasizing that the decoded values are not a perceptual fidelity measure like ΔE or a gloss value. The author concludes by providing instructions on how to reproduce the calculations using Python and Pillow, emphasizing the need to avoid conversions, resizing, or screenshots from the article page to maintain the integrity of the analysis.