Git repository hosting at scale presents challenges because Git’s design treats every repository copy as identical.
Git repository hosting at scale presents challenges because Git’s design treats every repository copy as identical. Linus Torvalds created Git to replace BitKeeper for the Linux kernel, a project with a decentralized workflow. Most open‑source projects and companies rely on a centralized host, which makes hosting a Git repository difficult.
The difficulty stems from Git’s use of packfiles, which are binary bundles of objects that must be stored on disk and transferred over the network. Packfiles are large, and their random layout makes operations that require reading the commit graph inefficient on networked filesystems. This issue affects both storage and network performance.
Early attempts to scale Git included distributing the filesystem, but networked filesystems such as NFS conflict with Git’s assumptions about file locking and performance. Later approaches that replicated the filesystem at the block level, such as GFS and DRBD, proved hard to operate and did not improve performance.
GitHub introduced a system called Spokes in 2013. Spokes stores repository data on local NVMe drives and uses three‑phase commit consensus to keep multiple replicas consistent. The approach limits scalability because each repository requires three replicas, and the consensus latency grows with the number of replicas. Spokes also treats each repository as a critical resource, requiring a routing table that maps repositories to servers and constant checksum verification, which adds operational complexity.
In 2026, the typical enterprise repository has become a large monorepo, and the three‑replica model is insufficient for high‑traffic CI workloads. Adding more replicas degrades push throughput because each step of the three‑phase commit is limited by the slowest node.
Continuity, a system built by Cursor, addresses these issues. It uses a write‑ahead log stored in S3‑compatible object storage as the source of truth. When a push occurs, the packfile is written to local NVMe storage and uploaded to S3 before acknowledgment. The push becomes visible only after the reference transaction is prepared locally and a pointer to the WAL entry is recorded, ensuring linearizable behavior.
Continuity’s design treats repositories as transient caches; the location of a repository does not matter. Repositories are mapped to nodes using rendezvous hashing, and any node can serve a repository. Updates are coordinated through an atomic compare‑and‑swap on S3, eliminating the need for a fixed primary or complex consensus.
Replication in Continuity is optimistic and relies on UDP gossip to propagate metadata. Reads verify the ETag of the latest WAL index against S3, achieving strong consistency without distributed locks. This allows a variable number of replicas per repository, from a single replica for idle repositories to many replicas for large monorepos, enabling linear scaling of read performance.
Compaction of packfiles is performed only by the primary node, with results propagated through the WAL, reducing CPU load on replicas. Synthetic tests have shown linear read scaling with up to 100 replicas and push throughput of 120 pushes per second on standard S3, with higher rates possible on S3 Express One Zone.
Azure DevOps stores packfiles in blob storage and uses a relational database for references, illustrating alternative trade‑offs. Continuity’s WAL‑first approach prioritizes Git data consistency over other considerations, avoiding the need for an external database.
The system aims to provide smooth migration paths, higher reliability, and improved performance for Git hosting, addressing the productivity costs associated with repository downtime.
- Publisher
- Hacker News
- Reliability
- high
- Published
- 8/21/2026, 10:00:25 AM
- Retrieved
- 8/21/2026, 10:00:25 AM
- Relevance
- 80%
- Confidence
- 85%

