Urgent.News

What's breaking now, across thousands of outlets.

Tech

Microsoft outage: A cascading failure that started with an undetected bug

Following a roughly day-long global outage of many Microsoft services last week, Microsoft on Monday issued a preliminary post-incident report that identified multiple causes for the issues, including the admission that the environment used to test the underlying problematic software update did not match the actual infrastructure it would run on. The report, a summary of which was posted on…

Microsoft outage: A cascading failure that started with an undetected bug

Last week, Microsoft suffered a global outage of many of its services that lasted over a day. In a post-incident report, Microsoft identified multiple causes for the issue, admitting that the testing environment used for a problematic software update did not accurately reflect the actual infrastructure it would run on. A latent defect caused servers to become overwhelmed and unresponsive under certain load conditions, resulting from a fix deployed for Microsoft 365 networking gateway servers in July.

Microsoft's test environment did not detect this issue because it did not match real-world conditions, and the increased load due to unrelated traffic compounded the problem. The combination of factors led to a cascading failure, with servers becoming unresponsive in less than two hours and over 80% of servers unresponsive at their lowest point.

Despite Microsoft's attempts to restore service, its recovery processes were not designed to handle this broad unresponsive state. Experts agree that Microsoft needs to better explain the discrepancy between its testing environment and production infrastructure to avoid similar issues in the future.

Written by urgent.news from Computerworld's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at computerworld.com →

More in Tech

More from Monday 16 November →