OpenAI built itself a new Git backend, too
OpenAI on Monday detailed how it built a new Git backend for its monorepo to power the ever-increasing number of The post OpenAI built itself a new Git backend, too appeared first on The New Stack .
OpenAI unveiled a new Git backend named Pando on Monday, designed to handle the surging number of commits from its developers and AI agents. The new backend slashed push latency by nearly 5 times and currently processes near a petabyte of Git data daily at OpenAI. This follows an explosion in quarterly commits to its monorepo, up over 2,400% since the first quarter of 2024.
OpenAI's challenges with its monorepo are akin to those faced by other enterprises and Git platforms like GitHub, which are grappling with Git's capacity to handle the accelerating pace of development. Pando aims to maintain compatibility with Git while addressing its limitations. The team sought to retain Git compatibility due to reliability issues with their existing backend, with fetches falling below 90% and slow merges.
OpenAI's Aaron Friel, Taylor Blau, and Ted Nyman questioned whether they had reached a limit of Git or the way it served their monorepo. Pando eliminates coordination across the whole repository during pushes, handling each branch independently, which prevents unrelated pushes from waiting on each other. However, this approach may result in slightly different branch states being visible to clients.
When a client fetches, Pando selects the required packs based on the client's history, serving them through regional caches with a hit rate of over 99%. The team implemented Pando in Rust in just 10 weeks, with one engineer and teams of Codex agents. OpenAI ships its own Git build for developer machines and the ChatGPT desktop app.
Written by urgent.news from The New Stack's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
