A fallback that writes a file must never report success

When creating a fallback mechanism that generates files, it is crucial to ensure that success is never falsely reported. This issue became evident when the NoCodeExport tool, which converts websites built with platforms like Framer, Webflow, Wix or Squarespace into static HTML for easy hosting, encountered several real failures. These failures were initially reported as successes, leading to a total of ten degraded pages despite the system indicating a successful export.

The first failure involved a Webflow export where ten pages were exported but each page had fallen back to the raw captured DOM. The pipeline processed each page without error and reported a success status, with all ten pages showing "Pages: 10". However, all these pages were degraded. The problem was that the fallback mechanism did not differentiate between the success status and the degraded state, resulting in an incorrect report.

To address this issue, the fallback mechanism was updated to return additional information in the output. The updated output includes a boolean flag "degraded" indicating whether the page was degraded due to the fallback mechanism. This flag allows downstream processes to distinguish between a successful export where all pages are functional and an export with degraded pages that were handled using the fallback mechanism.

The second failure involved a Wix export where six pages failed to scroll, all reported as successful. The pages were shipped with a specific class "responsive siteScrollingBlocked" added by Wix to block page scrolling. However, the fallback mechanism did not detect this issue, leading to a false success report. The issue was resolved by adding a check for the presence of the "siteScrollingBlocked" class and reporting it as an error if it exists.

The third failure involved Webflow sites where specific signals (e.g., "squarespace-credit", "generator meta") were not being detected correctly. The fallback mechanism relied on these signals to determine the platform that built the page and execute platform-specific steps accordingly. However, the detection mechanism was flawed, leading to silent switching off of platform-specific steps.

To fix this issue, the fallback mechanism was updated to use alternative signals (e.g., "Squarespace.afterBodyLoad") that were guaranteed to work across all platforms. This ensured that the fallback mechanism could accurately identify the platform and execute the appropriate platform-specific steps.

Lastly, the fourth failure was related to the author's own instrument used for measuring the performance of Webflow exports. The instrument reported that a specific scroll animation was reproduced in 1 out of 27 pages, which was unexpected. The author initially suspected that Webflow's motion engine had outrun the instrument. However, upon checking the instrument itself, it was discovered that the issue was a false positive caused by a narrow browser window.

The author learned that when a measurement yields surprising results, it is essential to first verify the instrument being used before attributing the issue to the code.

In summary, when implementing a fallback mechanism that generates files, it is crucial to ensure that success is never falsely reported. This involves reporting additional information about the export status, detecting and handling platform-specific issues, and verifying the accuracy of measurement instruments to avoid false positives.