Top 3 Data Center Outages of 2015
As another year comes to an end, and plans begin to develop for 2016, we thought that we would close the year reviewing the top 3 business outages of 2015. Data center downtime is one of those nightmarish scenarios that we all hope and pray won’t happen to our business, as we watch it happen to other companies and thank the BC/DR gods it’s not us. However, this attitude often leads to the common misconception that data center downtime won’t happen to us.
We could quote you all the statistics from previous studies talking about how 76% of those polled have experience downtime at some point in a year. But we thought some hard examples would really bring home the message that everyone is susceptible, and the effects are painful.
3) FAA is OOO (Out of Office) – We discussed this one a bit as it was happening, but a software update caused the Federal Aviation Administration (FAA) systems to go down for about 3 hours. This downtime essentially shut down air travel along the US Eastern seaboard for this time period and more. The ripple effect of this though was much bigger. Aside from thousands of delays, over 450 flights cancelled, travelers were left in the dark about their situation, with many not making it to their destination for up to two days.
2) Facebook’s Hat Trick – Generally hat tricks are good, but Facebook caught the negative side of the outage hat trick as they experienced 3 outages in 1 month! Aside from those who enjoy frequenting the social network, there were serious business damages. Developers were unable to access Facebook data, which prevented many users from being able to access mobile apps from with their Facebook accounts (i.e. Tinder). The outages also effected companies that depend on Facebook for their operations and products. Additionally, many publishers lost notable traffic, since Facebook is an important source of traffic.
1) No PayPal For you – This outage might have been the easiest to quantify this year in terms of dollars and cents and it was scary! A problem in one of PayPal’s data centers shut down the payment service for just about two hours. At first glance that does not make your jaw drop, but PayPal processes around 645 million dollars per day, more than 26 million dollars per hour. This outage equals over 50 million dollars of non-processed payments and the result is lost revenue for PayPal.
These three large organization all felt the pain of data center outages that could have been avoided had the correct procedures been put into place; first – replication with RPOs of seconds and RTOs of minutes, and second – regular, frequent testing of the BC/DR system. So as we enter 2016, learn from the mistakes of 2015 and start the year off right.