{
  "id": 9660489,
  "title": "Internal linking as a data structure",
  "url": "https://urgent.news/2026/09/25/internal-linking-as-a-data-structure",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-25T00:01:05.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/nooralto1/internal-linking-as-a-data-structure-16ej"
  },
  "original_language": "en",
  "account": "Internal linking functions as a data structure within websites. Initially, editors manually insert links, which becomes unwieldy as the site grows. To address this, internal links should be treated as a graph, where each URL is a node and 'a href' pointing to another URL on the same domain is a directed edge. This allows for computing orphan pages, link depth, and missing connections programmatically.\n\nTo build the node list, the sitemap serves as the fastest source. Parse the sitemap to obtain the full set of URLs, then crawl the HTML to extract the edges. The XMLParser module helps in parsing the sitemap XML, and the fetch function is used to retrieve the URLs from the sitemap. If the sitemap contains other sitemaps, they are recursively parsed.\n\nThe edge list is built from the HTML of each URL. Cheerio library is used to parse the HTML and extract all internal anchor tags. The URLs are then normalized and added to a set to avoid duplicates. The extracted links are stored in an adjacency table, where each URL is a key and its corresponding value is an array of URLs it links to.\n\nDepth of a page, often used as a proxy for authority, can be computed using a breadth-first search algorithm. The computeDepth function takes the adjacency table and a root URL as input. It initializes a depth map with the root page at depth 0 and a queue with the root page. The algorithm iteratively dequeues a page, checks its adjacent pages, and enqueues them if they haven't been visited yet. The depth of each page is stored in the depth map.",
  "summary": "Internal linking as a data structure Most internal linking work happens by hand: an editor remembers a related page and drops a link at the bottom of a post. That works until the site passes a few hundred URLs, at which point nobody remembers what links to what, and pages start going dark without anyone noticing. The fix is to stop treating internal links as content and start treating them as a…",
  "key_points": [
    "Internal linking functions as a data structure within websites",
    "Treat internal links as a graph with URLs as nodes and hrefs as directed edges",
    "Compute orphan pages, link depth, and missing connections programmatically"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}