GitHub restores services after nearly 8-hour outage disrupts Actions, APIs, PRs and Copilot

GitHub has resumed its services following a prolonged eight-hour outage that disrupted a multitude of its essential developer tools, including Actions, pull requests, APIs, Git operations, Webhooks, and Copilot. The company informed its users via the status page that the issue had been rectified. The disruption was initially spotted at 1:40 PM UTC on August 17, 2023.

At its peak, GitHub documented an error rate of approximately 20% for its web experience and API traffic. Archive downloads and raw repository content downloads had an error rate of around 50%. Additionally, authentication-related issues affected SAML and OIDC, SCIM, and Team Sync. GitHub's AI coding assistant, Copilot, also experienced degraded availability, beginning at 2:31 PM UTC.

Recovery was not straightforward, with various services intermittently encountering further performance issues. Some Git Operations issues resurfaced, and API Requests temporarily regressed to a degraded state. GitHub attempted to mitigate the Git Operations problem, and restored normal API operations by 7:01 PM UTC. Despite these efforts, authentication-related issues persisted for some applications.

The company partially disabled authentication-token retries, observing a decrease in sporadic authentication failures. They reported resolving these issues at 8:45 PM UTC, marking the incident as resolved at 9:15 PM UTC, nearly eight hours after it began. The outage's impact on enterprise teams is significant, as the affected services are integral to their software development workflows.

GitHub is yet to pinpoint the exact root cause, leaving the incident's origin unclear. The company has promised to share a detailed root cause analysis once available.