{
  "id": 15163457,
  "title": "Developing for Debugging (2025)",
  "url": "https://urgent.news/2026/10/17/developing-for-debugging-2025",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-17T07:21:14.000Z",
  "source": {
    "name": "Lobsters",
    "slug": "lobsters",
    "url": "https://www.davepacheco.net/blog/2025/developing-for-debugging-dns-propagation/"
  },
  "original_language": "en",
  "account": "Oxide’s control plane manages two DNS services: external DNS accessible to customers for locating the system’s APIs and web console, and internal DNS for internal network service discovery. The authoritative source of truth for DNS data is a strongly consistent distributed database called CockroachDB. Nexus, a component within the control plane, updates the DNS configuration in the database when changes occur, such as adding or removing components that provide service internally or externally.\n\nThis process involves background tasks in Nexus that propagate the updated configuration to all in-service DNS servers. These tasks run asynchronously after the event triggering the change but usually complete in a few seconds. While designing this system, the team recognized the need for better visibility into the DNS propagation process to debug issues when DNS names are missing. They built omdb, an internal debugger for the control plane, which provides observability into the DNS propagation process.\n\nOmdb tracks key information about DNS configuration, such as what names are part of the configuration, when it was last updated, and by which Nexus instance. For each Nexus instance, it reports the version of the DNS configuration last propagated and the DNS servers it attempted to propagate to. For each DNS server, omdb provides details about the version of DNS data it is operating from.\n\nWith these insights, the debugger can generate a comprehensive summary report on DNS propagation, allowing developers to pinpoint where the pipeline may have broken down. This level of detail goes beyond simply reporting \"no problems\" or \"up to date,\" as it provides concrete evidence that the system is functioning correctly. Such information can help determine if DNS is the issue or if there are other underlying problems.\n\nThe omdb debugger not only aids in real-time debugging but also during development. It enables developers to test and play with the system before writing automated tests and understand how components interact. By using omdb, teams can create better onboarding documentation, train new team members, and provide live demos to illustrate how the system operates. Developing with a focus on debuggability ensures that the system is not only functional but also easy to troubleshoot and maintain.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}