Running ML models in the browser with ONNX Runtime Web: a practical guide
FaceVision does face detection, recognition and liveness checks without sending a single frame to a server, because every model runs in the browser through ONNX Runtime Web. That architecture is the reason its privacy story is simple. It is also the part students find hardest to get working, so here is the path that worked. Export, then verify Export the model to ONNX from its training framework…
FaceVision enables face detection, recognition and liveness checks without sending any data to a server, as all models run in the browser through ONNX Runtime Web. This architecture simplifies the privacy story, while also posing challenges for students attempting to get it working. To address this, here is the step-by-step guide that successfully implemented the solution.
First, export the model to ONNX from its training framework and immediately compare the output with the original graph and reference implementation. This verification process ensures the model is functioning correctly. The source reveals that many reported issues were due to pre-processing mismatches, which could have been easily caught with a simple comparison.
Next, choose the execution provider with care. WebGPU is the fastest option when available, and it should be used unless alternative options are needed. WASM with SIMD and threads are suitable as a reliable default, but they require the right headers for multi-threading. WebGL, although a legacy option, is not recommended for new work.
Proper pre- and post-processing is crucial to prevent bugs. Models require specific channel orders, normalisation, and input sizes. Images arrive in RGBA bytes in a browser, so it's essential to write the conversion once, test it against a known image, and keep the output tensor shapes in a comment near the code. Post-processing tasks, such as decoding anchors for a detector or normalising an embedding before cosine similarity, also demand careful attention.
Performance should feel "live". To achieve this, warm up the session with a dummy inference on page load, as the first run is always slow. Utilise Web Workers to run inference, ensuring the camera preview never stutters. Additionally, downscale frames before detection and only run recognition on detected crops. When the queue is full, skipping frames instead of allowing latency to grow can prevent issues.
In FaceVision, the FastAPI and PostgreSQL backend stores only vector embeddings and matches them during enrolment and verification. No images are ever persisted, making the privacy claim simple to explain and audit. This design choice is detailed in Rathour's blog post "Face recognition that never uploads a face".
Browser-side inference is not a gimmick; for applications involving faces, documents, or health data, it often makes the product acceptable to users. Pranjul Rathour, a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, presented this solution at an Integral Startup Foundation hackathon, a packed college auditorium, and a formal campus event.
Now, he is a mentor to over 200 students through TechVerse Enclave, judges and speaks at student hackathons across India. For inquiries about GenAI, campus sessions, or to invite him for judging or speaking engagements, Rathour can be reached at pranjulrathour41@gmail.com.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
