ENAMETOOLONG: how one dirty database field crashed our static build

At podbor-minuta.ru, approximately 600 apartment listing pages are generated beforehand as static HTML. Each page's URL is constructed using city, district, room count, and price data. However, a single field containing an overly lengthy string led to a build failure.

During the build process, the generator took the district field from the database and incorporated it into the URL. The static builder (Vike) created a directory for each address on the disk. Unfortunately, directory names in the filesystem cannot exceed 255 bytes. In this case, the district field contained a 295-character sentence instead of a proper district name, causing the directory creation to fail.

Initially, the issue remained undetected as the build utilized Docker and cached the static-build layer, allowing the step to be skipped for a brief period. However, when executed locally, the build occasionally reached the problematic step due to data fetching delays, resulting in an incomplete page not being created. The failure became apparent only during clean builds in the continuous integration (CI) environment.

To remedy the situation, two key changes were implemented. Firstly, a safety check was introduced within the address generator function. Any address segment longer than a reasonable limit of 120 characters was discarded, preventing the creation of problematic pages. Real district and station names are typically shorter than 120 characters.

Secondly, the source was cleaned by filtering out undesirable data. The generator no longer accepted district values that appeared to be long description sentences, containing digits or words like "rubles," "apartments," or "finishing." By removing such entries, the generator avoided generating numerous thin, empty pages.

The crucial takeaway from this incident is that any data originating from uncontrolled sources, such as scraped data or user input, must undergo thorough validation and trimming before being used in file names or paths within a static site. Ignoring such data can lead to catastrophic failures, making it essential to ensure that each piece of data is properly checked and cleaned to prevent entire builds from crashing.