Your page has two readers: humans, and machines that never run your JavaScript
I keep running into the same bug, and it isn't in anyone's code. It's in the gap between your site and the machines that describe it to other humans. When someone pastes your link into Slack, iMessage, X, or LinkedIn, they don't see your page. They see whatever a bot pulled from your HTML server-side: no JavaScript, no rendering, no images loading late. Same for a lot of crawlers, and for every…
When a link to your website is pasted into communication platforms like Slack, iMessage, X, or LinkedIn, humans typically do not receive the full experience of your site. Instead, they see a version created by a bot that accessed your HTML server-side, devoid of JavaScript, rendered text, and images. This phenomenon also applies to web crawlers and scrapers that interact with your site.
Consequently, your website has two distinct audiences: human users and machines. To better understand how machines perceive your site, you can quickly compare two layers of your webpage. The first layer represents the rendered text that headless browsers and reader APIs extract after your page loads. This layer closely resembles what a search engine would see.
The second layer is the raw HTML document exactly as it was sent from the server, with JavaScript disabled. This layer is accessible to link previews and simple scrapers. By examining both layers side by side, you can identify discrepancies and potential issues. For instance, the absence of a title tag, meta description, or og:title, og:image, and twitter:card tags can result in poor link previews and poorly formatted scrapes.
If your content relies on JavaScript to populate headlines, copy, or forms, the raw layer will likely be empty or significantly different from the rendered layer. This discrepancy often leads to search engines improvising with default information, such as a domain name or a scraped fragment, which can negatively impact your site's visibility and user experience.
To address these issues, ensure that your server-side sends the necessary metadata and consider implementing server-side rendering (SSR) or static site generation (SSG) for bots that require content extraction. You can utilize a free online tool that demonstrates how machines view your website by comparing the rendered and raw layers for any missing elements like titles, descriptions, og tags, viewport settings, and empty raw HTML.
While this tool cannot predict human behavior or identify subtle user interactions, it serves as a valuable starting point for improving your site's accessibility to both human readers and machines.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.