Three iOS WebKit Traps That Blacked Out My AR Game, and How I Caught Them
AURADUEL's camera went black on my iPhone in three ways. The evidence, the two fixes that failed, and the rule that worked: one video element per stream.
Three iOS WebKit bugs caused an AR game to black out. The game used a hidden getUserMedia call, pixi.js for WebGL rendering, and MediaPipe for body and hand tracking. All ran inside a Capacitor app's WKWebView, meaning WebKit.
In the first bug, switching cameras caused the main thread to freeze, making the watchdog think the camera was paused. Two fixes were added: the watchdog now has its own heartbeat and a main-thread heartbeat to detect JavaScript blockage.
The second bug occurred when unmounting a React scene during a camera switch. The scene destroyed its pixi scene and video texture, leaving the camera element and detector using an unusable video texture. The fix was to keep the arena mounted through transient camera states and apply live mirroring or reframing.
The third and most significant bug involved a reused pixi scene texture. After switching cameras, the detector saw the hands through the black canvas. MediaPipe creates its texture from the element on every frame, but the scene texture was still tied to the previous stream. The fix was to ensure the texture was properly updated when switching camera sources.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
