{
  "id": 14719541,
  "title": "Everyone is doing Git on S3 wrong",
  "url": "https://urgent.news/2026/10/15/everyone-is-doing-git-on-s3-wrong",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-15T09:07:04.000Z",
  "source": {
    "name": "Lobsters",
    "slug": "lobsters",
    "url": "https://enroute.sh/blog/everyone-is-doing-git-on-s3-wrong"
  },
  "original_language": "en",
  "account": "The article discusses the limitations of Git hosting solutions that rely on object storage, specifically Amazon S3. Traditionally, Git hosting has used SSD disks, which provide fast random access storage, but this comes at the cost of potential disk failures, the need for RAID and backups, and eventually running out of disk space. Object storage, like S3, has become more reliable and offers durability, automatic replication, and unlimited capacity. This has led to efforts to move Git repositories onto object storage.\n\nOne notable attempt is Cursor/SpaceX's Continuity, which utilizes S3-backed Git storage, specifically an S3-backed write-ahead log (WAL) for Git. Other open-source implementations have followed a similar approach, moving scalability and durability to object storage. However, these implementations still rely on the canonical Git CLI, which memory maps files on disk. This means that each repository must be fully loaded on disk before it can be served, resulting in write-heavy traffic causing performance issues.\n\nThe article outlines the challenges Git faces when stored on object storage. Git is fundamentally a content-addressed storage system with mutable pointers, which allows objects to point to any other object and enables efficient querying. However, treating Git as just content-addressed storage on object storage would result in each step being a 30-40ms round trip, making it unsuitable for Git.\n\nTo address these challenges, Cursor/SpaceX's Continuity design stores Git repositories as immutable packfiles and uses a WAL index to log each push. Packfiles are served through conditional PUTs against the WAL index, enabling any machine to rebuild a local copy from the log and serve fetches. This approach scales reads well, with Cursor reporting linear read scaling up to 100 servers. However, the write path still depends on a single linearizable sequence of packfiles, causing competing write operations to contend for the same WAL index object, which can be bottlenecked by the speed of Git's disk compaction. While the Continuity design significantly raises the ceiling for Git hosting, it still falls short of making Git fully scalable, as it remains a single-writer system.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}