Your GAN Beauty Effect Needs a Device Budget, Not a Universal “On” Switch

In the pursuit of creating realistic beauty effects through GAN-powered technology, developers must be mindful of the device budget constraints that exist in real-world scenarios. It is not merely a matter of whether the effect is labeled as "AI" or not; rather, the critical question lies in determining how various components such as appearance processing, segmentation, rendering, and video should be allocated when competing for limited device resources.

Simply enabling every effect without consideration can lead to performance issues on lower-capability devices.

The key to tackling this challenge lies in defining clear rules for consent, capability, fallback, and verification. These rules should remain comprehensible even in situations where the renderer fails. Instead of merely adding toggles to an application, the focus should be on establishing a well-defined policy that ensures the correct behavior of the GAN beauty effect under various conditions.

In this tutorial, we will explore the development of an application-owned Beauty AR controller that takes into account the following factors:

1. It will not process appearance effects before obtaining consent from the user.

2. Both GAN and other expensive effects will be treated as optional capabilities.

3. The system will select the appropriate profile based on measured device evidence.

4. In case of sustained frame pressure, the system will downscale the profile rather than reacting to a single noisy sample.

5. The application will refuse to auto-upgrade during an active session.

6. It will ignore stale asynchronous callbacks.

7. If even the safest profile cannot be applied, the effects will be disabled.

Tencent RTC Beauty AR offers a range of scenarios, including real-time beauty filters, makeup, stickers, virtual backgrounds, avatars, gesture recognition, and image or video enhancement. For further information on the available features, refer to the official Tencent RTC Beauty AR overview at https://trtc.io/document/beauty-ar-overview.

Additionally, Tencent RTC's low-end optimization guide provides recommendations on adapting configuration to device capabilities, utilizing performance-oriented modes, controlling resolution and frame rate, and disabling resource-intensive segmentation or 3D/GAN effects when necessary at https://trtc.io/document/66968.

The provided code below serves as a starting point for implementing an application policy, rather than a complete replacement for the official platform-specific integration instructions. It separates the user's visual contract into three distinct decisions: the application's ability to alter or process the user's appearance (consent decision), the user's preferences for specific effects (preference decision), and the effects that the current device can sustain (operational decision).

The code defines three states: awaiting-consent, granted, and probing. Depending on the state, the application may enter the applying phase or proceed to running with sustained pressure or degradation. If the system fails to apply a safe profile, it will revert to the off state or degraded state, depending on the severity of the failure.

To facilitate testing, the project has been structured using Node.js with TypeScript. The necessary scripts have been added to the package.json file to enable testing without a camera, live room, or specific Beauty AR SDK methods. The policy can be tested by representing visual cost as defined profiles, ensuring that the policy operates independently of the renderer callbacks.