When datacenters lose power, generators should kick in. That didn’t happen for Google in London

On October 20, Google Cloud experienced an outage in its europe-west-2-b availability zone in London due to a "misconfiguration" of its systems, which prevented its emergency generators from activating. The incident disrupted services for major organizations such as Lloyds, HSBC, Royal Mail, and Revolut. In a November 4 update, Google explained that the issue stemmed from a 2N utility power architecture, where the facility has duplicate power paths for all critical loads, backed by a central high-voltage generator bank managed by a dual-redundant Programmable Logic Controller (PLC).

The facility relies on two independent utility feeders, labeled 'A' and 'B', along with a standby utility connection that can be manually activated for maintenance or emergency events.

A break in an external underground cable, connecting to feeder 'A', coupled with feeder 'B' being physically opened for repairs following a quarterly generator load test on October 14, 2026, meant that primary utility feed 'A' supported the total critical load while feeder 'B' remained open. This setup, combined with a voltage-sensing design flaw and PLC misconfiguration, prevented the facility's generator plant from receiving an automatic start signal, causing the UPSes to run out of power.

Despite the UPSes functioning correctly, the failure to start the generators resulted in the outage. Google recognized the PLCs' lack of proper configuration for the circumstances faced that day, reprogramming them to avoid a similar mistake. Following the incident, Google implemented several changes to the electrical setup at europe-west-2-b, including conducting tests and implementing enhanced communication mechanisms and broader platform-wide testing for infrastructure-level events.