Keeping noindex, the sitemap and the image sitemap in agreement for programmatic pages
Google's documentation for sitemaps says to "include the URLs in your sitemap that you want to see in Google's search results". Its documentation for the robots meta tag says the noindex rule means: "Do not show this page, media, or resource in search results." A URL that carries both is giving two answers, and when the pages are generated from one array it is easy for them to drift apart without…
Google's documentation for sitemaps and the robots meta tag both advise including URLs in sitemaps that you want to appear in Google's search results, while the noindex rule directs search engines not to display the page in search results. When pages are generated from a single array, the sitemap.xml and image-sitemap.xml files can become out of sync, especially for pages with thin content.
In the case of the landscape design website Landscapecraft.ai, there are 40 design-style pages generated from one TypeScript array. If a style lacks a long-form MDX write-up, it is treated as thin and assigned noindex, follow. As a result, such pages are absent from both sitemap.xml and image-sitemap.xml files.
The code maintains consistency between the three files (sitemap.ts, image-sitemap.xml/route.ts, and app/styles/[slug]/page.tsx) by relying on a single predicate. When one of these files detects a page without a long-form MDX, it flags it for exclusion from all three sources.
However, these three files are not perfectly aligned. For instance, idea pages with fewer than 3 posts are still indexable and included in sitemap.xml. Additionally, the image sitemap does not list category pages at all, further causing discrepancies. While the filter only covers one page type, it does not account for variations in page type, leading to inconsistencies that require monitoring and adjustment.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
