{
  "id": 4260371,
  "title": "Seven Ideas That Keep Distributed Systems From Falling Over",
  "url": "https://urgent.news/2026/08/29/seven-ideas-that-keep-distributed-systems-from-falling-over",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-29T19:09:10.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/lovestaco/seven-ideas-that-keep-distributed-systems-from-falling-over-2nbf"
  },
  "original_language": "en",
  "account": "Hello, I'm Maneshwar, and I'm building LiveReview — an AI code review that's aware of potential blast radius for your business-critical systems. Star us to help developers discover the project, give it a try, and share your feedback to improve the product. While we may not have failures, everyone has them. Networks partition, disks fail, servers get evicted without explanation. Reliability isn't about preventing failures; it's about the system still doing the right thing when they occur. There are seven ideas that keep recurring when examining how large systems maintain reliability. After researching, a few ideas directly connect to Kademlia and XOR distance, which I previously wrote about.\n\n1. The CAP Theorem: a distributed system can provide at most two out of these three: Consistency, Availability, and Partition tolerance. Consistency means every node sees the same data simultaneously, Availability ensures every request to a non-failing node receives a response, and Partition tolerance means the system keeps running even when the network splits. However, partitions are not optional in networked life, as cables can cut, switches die, and cloud regions lose connectivity. So, CAP essentially becomes \"choose consistency or availability for the duration of the partition.\"\n\nTwo real-world examples demonstrate different approaches to handling partitions:\n- Google Spanner prioritizes consistency. It employs atomic clocks and GPS-synced time to maintain linearizable transactions across continents. During a partition, the majority side continues serving reads and writes, while the minority side transitions to read-only until the situation resolves.\n- Amazon DynamoDB prioritizes availability. It accepts writes during a partition and resolves conflicting updates afterward using timestamps. It always provides a response, although it may be slightly outdated.\n\nNeither approach is inherently wrong; it depends on the specific requirements. A bank ledger would prefer Spanner's consistency to ensure accuracy, while a Twitter feed doesn't need to be linearizable, as minimal downtime is acceptable.\n\nConsistency on the wire means every node holds the exact same value simultaneously. This guarantee comes at a cost, which is why not all systems implement it. Now let's examine availability. In this scenario, the system prioritizes responding to requests over providing the most recent data. The older values (t-1, t-2) can still exist on some nodes. This trade-off allows every request to receive a response, even if it's a slightly outdated one. During a partition, Spanner would stop accepting writes and start rejecting them, while Dynamo would continue serving requests, albeit with potentially stale data.\n\nThese three factors—consistency, availability, and partition tolerance—guide the decision-making process when a partition occurs. Nodes make a choice based on their proximity to a majority quorum. If they can reach a quorum, normal operations resume. If not, they must either reject the request or serve possibly stale data. This decision-making process encapsulates the essence of the CAP theorem in practice.",
  "summary": "Hello, I'm Maneshwar, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product. How does Amazon stay up during Black Friday when a normal server would be on fire by 9am? Why do banks rarely lose a transaction even when a data center loses…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}