Wikipedia is wondering why its network provider’s resiliency measures didn’t work
Wikipedia was down for around an hour yesterday after a fibre cable between two of its data centres was cut.
For redundancy reasons, its two data centres – one in Ashburn, Virginia and one in Tampa, Florida – are connected by two separate fibre links. Ashburn serves most of Wikipedia’s traffic, but has to talk to the Tampa facility for backend infrastructure such as databases.
Despite having additional redundancy measures in place, with two separate 10G fibre cables running between the two data centres, the site still went down. Wikimedia is talking with its network provider to figure out why failovers didn’t work and keep Wikipedia alive.
“We are now working with our network provider to determine how and why we were impacted by that fiber cut when we are supposed to have redundancy in our network. We are still waiting for their full report,” said Wikimedia director of technical operations, CT Woo, in a blog post.
“The team worked around the outage by rerouting traffic to Tampa, bypassing the Ashburn site. Connectivity was restored at about 8:35am PDT to one of the provider’s network links. The second link was restored at about 11:30am PDT.
“However, we have not reverted traffic back to Ashburn yet until we are comfortable with their fix. The switch back to Ashburn from Tampa should not be apparent to users.”
The last time Wikipedia was inaccessible, it was no accident. On 18 January, Wikipedia was blacked out for 24 hours in protest against SOPA (Stop Online Piracy Act) and PIPA (Protect IP Act) – proposed US laws that were criticised for infringing on the freedom of the Internet.
Are you a Firefox enthusiast? Try our quiz!