A firmware upgrade failure at one of Microsoft’s datacenters caused the facility to overheat and shut down access to Outlook.com and Hotmail for some users earlier this week, Microsoft said in a blog post this morning explaining the cause of the problem.
The outage, which started on Tuesday afternoon and lasted until early Wednesday morning, came at an inopportune moment for Microsoft, as the company tries to attract new users to its Outlook.com service. The outage also affected a smaller number SkyDrive users.
Of course, Microsoft is hardly the only online service provider to ever suffer an outage.
Here’s the root cause analysis from the Microsoft post …
On the afternoon of the 12th, in one physical region of one of our datacenters, we performed our regular process of updating the firmware on a core part of our physical plant. This is an update that had been done successfully previously, but failed in this specific instance in an unexpected way. This failure resulted in a rapid and substantial temperature spike in the datacenter. This spike was significant enough before it was mitigated that it caused our safeguards to come in to place for a large number of servers in this part of the datacenter.
These safeguards prevented access to mailboxes housed on these servers and also prevented any other pieces of our infrastructure to automatically failover and allow continued access. This area of the datacenter houses parts of the Hotmail.com, Outlook.com, and SkyDrive infrastructure, and so some people trying to access those services were impacted.