Measuring faces in the browser with MediaPipe: three things that broke
I wanted a face-measurement tool with three properties most "face rating" sites don't have: the photo never leaves the browser, the same photo always gives the same numbers, and every formula is public. The result is FacialRatio , and its engine is open source . This post is about the parts that did not work the first time. The setup Landmarks: Google's MediaPipe Face Landmarker, loaded as…
The facial measurement tool FacialRatio was designed with three key properties: the photo never leaves the browser, the same photo always yields the same results, and all formulas are publicly available. This post discusses the issues encountered during its initial development.
The first problem was with level heads appearing tilted. Initially set to reject angles greater than ±8°, 7 out of 13 photos were rejected. The engine read level heads with a median pitch of +8.2°, even when they appeared level. The solution was to center the window on the offset produced by the model, changing the gate to −8° to +16°. The lesson learned was that landmark models have their own pose bias, which can only be detected by testing on real-world photos.
The second issue was asymmetry caused by a slight head turn. Symmetry was measured by mirroring landmarks across a fitted midline and averaging the distance to its counterpart. However, on real photos with heads turned 5–8°, the far side of the face was foreshortened, leading to an asymmetry report. To address this, the engine now estimates yaw, rotates the points back to frontal, and then measures symmetry. Faces turning beyond ±10° still trigger a retake instead of being corrected.
The third problem was the inability to accurately measure the hairline. MediaPipe's face mesh doesn't include a hairline point, making it impossible to measure facial thirds accurately. The tool uses the upper forehead point as the hairline, with the measurement method and limitations clearly stated in the results and methodology page. The golden ratio, being the most-searched feature, gets the lowest weight (15%) due to inconsistent evidence of its prevalence across different faces.
For improvement, it's suggested to collect a small set of everyday test photos at the start, document every constant in a single configuration file, and make the engine, tests, and canonical-face fixture available on GitHub. The full methodology, including known limitations, is available on the FacialRatio methodology page. Open feedback on the measurement approach is welcome.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
