How Cursor beat Git's scalability shortcomings

Developers who have experienced GitHub's system outages may find relief in alternative solutions for managing Git at scale. Cursor, a SpaceX subsidiary, has developed an approach by building its Git-based repository service, Origin, on object storage. Cursor principal systems engineer Vicent Martí explains how the team tackled the scaling issues of Git, a notoriously complex distributed version control system.

The article highlights the challenges posed by developers creating numerous small repositories and the difficulties Git faces with "eventual consistency." Cursor's solution involves object storage, specifically Amazon Web Services' Simple Storage Service (S3), as a foundational layer for enterprise software. Origin uses a write-ahead log (WAL) to capture all changes as immutable objects in S3, while simultaneously writing to a local NVMe disk reference copy.

This approach enables faster ingest of pushes and allows other replicas to download changes as needed.

While Git still requires DAG traversal for many operations, it is now better to perform these tasks locally on a solid-state drive rather than over a network. Cursor's Origin service can ingest pushes as fast as the disk allows, with the source of truth always being the write-ahead log.