I built a marble run physics sandbox in the browser, and the export was the hard part

I developed a marble run physics sandbox in the browser, a project that turned out to be a challenging endeavor. The website, accessible at https://www.marblerun.online, allows users to create 3D marble runs using ramps, walls, pegs, spinners, boxes, and balls on a lit wall. Simply press play, and a real-time rigid body simulation begins, culminating in a vertically formatted MP4 video complete with sound.

The entire process occurs client-side, without any registration, login, or data upload requirements, and it does not utilize any AI technology during runtime. The design is tailored for desktop browsers only, as the scene is rendered in 3D, and mobile devices receive a brief, suboptimal version.

The export feature, however, proved to be the most daunting aspect of the project, consuming three times more time compared to the editor itself. Initially, I considered using MediaRecorder on the canvas, which seemed like a straightforward solution. Yet, quickly realizing its limitations, I decided to abandon that approach. Screen capturing a WebGL canvas leads to incorrect aspect ratios, variable frame rates, and audio that needs to be merged separately, all of which detract from the immersive experience of a true 3D environment.

Instead, I opted to render the same scene from a camera's perspective through an offscreen pass at a 1080x1920 resolution, maintaining a consistent 30 frames per second. This offscreen pass ensures that the video encoder consistently records frames, guaranteeing that the simulation steps at a fixed, deterministic rate. Consequently, the same level produces identical videos each time it is exported.

For the audio component, I generated a click track aligned with the impacts, meticulously synchronized with the visual output, ensuring a seamless integration between the simulated actions and the resulting audio.

Several additional considerations contributed to the final outcome. Employing a fixed timestep with an accumulator ensures that the physics simulation remains consistent across devices. Running the physics independently of the requestAnimationFrame calls prevents discrepancies in gameplay between a 60Hz laptop and a 144Hz monitor.

For the marbles, substepped integration is crucial to prevent the frustrating issue of a fast marble appearing to pass through thin pegs, which would undermine the realism of the experience. Per-material restitution and friction settings allow users to customize the behavior of different materials, such as a steel ball on a wooden ramp versus a rubber ball on the same surface, introducing subtle yet essential variations to the gameplay.

Lighting plays a more significant role than anticipated. A single directional light with a soft fill and subtle textures on the wall are essential to prevent the scene from appearing flat and 2D. Even though the underlying geometry is genuinely three-dimensional, the lighting can dramatically affect the perceived dimensionality and visual appeal of the sandbox. Understanding these nuances was instrumental in refining the final product.

Reflecting on the process, I now recognize that starting with the export functionality would have been a more strategic approach from the outset. Had I initially focused on determinism—the key to ensuring consistent exports—I might have encountered fewer challenges during the development phase. Nonetheless, the final product is a testament to the intricate balance between physics simulation, rendering, and real-time audio syncing, all encapsulated within a single, immersive browser-based experience.