Railway Preliminary Report: Oct 21st Security Incident

On October 21st, Railway identified a security incident that resulted in unauthorized access to a portion of their internal systems. The company engaged external experts and authorities to assist in the investigation, which remains ongoing. Railway informed customers about the security issue and provided a timeline of events on October 22nd.

The attack began around 13:58 UTC on October 20th and concluded on October 22nd 03:32 UTC. During this period, customers experienced intermittent build and deployment failures, as well as periods of unavailability for dashboards, APIs, CLI, and support. As a result of the breach, a subset of customer data containing encrypted secrets was exposed, requiring customers to rotate their keys.

Railway's internal systems are designed so that if one set of keys is compromised, it should not endanger the entire system. Consequently, the company estimates that at most 1.85% of all workspaces had potentially compromised environment variables, affecting about 100,000 workspaces. To address this, Railway notified all customers who may have been impacted and proactively reached out to high-risk users. The attacker was seeking AI credentials, which led to the investigation focusing on high-value targets.

Railway worked with several partners, including GitHub, GitLab, Anthropic, OpenAI, OpenRouter, SpaceXSI, Stripe, and others, to revoke compromised keys as soon as the breach was confirmed on October 21st. Despite the disruption, all existing user deployments remained online and running throughout the incident. Railway recommends that all users revoke and reapply sensitive environment variables on their platforms prior to the recommended date.

The company has taken measures to enhance its security posture and protect customer safety. They continue to refine their platform and will remain transparent in their approach. For further information or questions, users can email support@railway.com. Railway emphasizes its commitment to operating with the same rigorous security standards that enabled the company to build its ambitious products.