How We Measure Whether an AI 3D Model Actually Matches Your Image

Measuring the accuracy of an AI 3D model in generating an image that matches the original input has evolved over the past two years. Instead of simply identifying obvious errors like holes or incorrect shapes, the current focus is on how closely the generated 3D asset matches the original object in terms of shape, structure, and surface details. This concept is known as geometry alignment.

To assess geometry alignment, Meshy 7, a new image-to-3D model, was evaluated against five other AI 3D models using a dedicated benchmark. Meshy 7 demonstrated superior performance in overall proportion alignment under single-view conditions, with scores significantly higher than its competitors. However, it excelled particularly in surface details, which often determines whether a model accurately reproduces an object or merely resembles it.

The measurement process involved comparing generated models against reference 3D models that were held out of the training data. By rendering images from the reference models using known cameras and feeding those images into each tested model, a known correct answer was established for each test case. The alignment was measured using translation, rotation, and uniform scaling, excluding stretching to prevent models from cheating by artificially improving in certain axes.

Three dimensions of alignment were evaluated: overall proportion, spatial distribution, and surface details. Overall proportion assesses whether the generated model occupies the same region of space as the reference, while spatial distribution measures how far generated geometry has to move to match the reference. Surface details evaluate whether the generated surface matches the reference image, including missing or altered features like bumps, patterns, or bolts.

Meshy 7's improvement across these metrics stems from three key changes in its development. First, the image encoder was rebuilt to read features at multiple scales and accept higher-resolution images, preserving finer shape information from the input image. Second, the training data was rebuilt using a stricter standard, removing the effects of style, lighting, and background to prevent the model from learning shortcuts.

Third, alignment was made a direct evaluation signal throughout development, ensuring that the capability improved during the training process.

The results indicate that Meshy 7's focus on accuracy in surface details, even with a single reference image, sets it apart from its competitors. While other models show improved performance with additional views, Meshy 7 maintains a significant lead in surface detail alignment, particularly in single-view conditions. This highlights the significance of capturing fine structural details in AI 3D models to ensure accurate and realistic representations of the original objects.