llms.txt on a 4-language static site: how we generate it, validate it and keep it honest

The article "llms.txt on a 4-language static site: how we generate it, validate it and keep it honest" discusses how a small team manages AI assistant interactions on their website, veilvex.com, which is available in English, Russian, Azerbaijani, and German. The team treats the website's robots.txt file as part of the build process to explicitly allow AI crawlers to access the site.

They generate a robots.txt file at build time and list specific AI bot names, with a Content-Signal line to indicate how the content may be used. The team also creates an llms.txt file, which is a fact sheet for language models explaining the site's purpose, services, and contact information. The llms.txt file is generated from the same i18n content files used to render the pages, ensuring consistency across languages.

The article emphasizes the importance of validating the llms.txt file during the build process to prevent inconsistencies and misleading information. The team uses a script that throws an error if the llms.txt file grows too large, price information does not match, or if forbidden claims are present. They also ensure that structured data, such as Service and FAQPage JSON-LD, agrees with the visible text and uses stable @id values across languages.

The organization node uses the same brand line as llms.txt. The team periodically checks which AI models are naming their website using ChatGPT, Perplexity, and Gemini and logs the results. The checklist includes allowing AI crawlers in robots.txt, having one page per service with prices and FAQs, generating llms.txt from content, failing the build on price mismatches or forbidden claims, using real lastmod dates, and checking fixed queries monthly.