The Xen Vulnerability That Rebooted the Public Cloud

Cloud computing © Sashkin Shutterstock 2012

More details are now public on the open-source Xen hypervisor vulnerability that triggered full Amazon, Rackspace and IBM cloud reboots

A proverbial lynchpin holds the world’s major public cloud providers together, and that pin is the open-source Xen hypervisor. Amazon, Rackspace and IBM SoftLayer have all had to reboot their servers in the last several days to fix a flaw in Xen that was privately reported two weeks ago and only publicly disclosed on Oct. 1.

The flaw in question is detailed in Xen Security Advisory XSA-108 and is also identified as CVE-2014-7188. Technically speaking, the vulnerability is titled “Improper MSR range used for x2APIC emulation,” which is basically a memory-related issue. Model Specific Registers (MSRs) are control registers within an x86 chip, while x2APIC is Intel’s next-generation Advanced Programmable Interrupt Controller (APIC).

“The MSR range specified for APIC use in the x2APIC access model spans 256 MSRs,” according to the Xen advisory. “Hypervisor code emulating read and write accesses to these MSRs erroneously covered 1024 MSRs.”

The impact of the flaw is that an attacker could potentially crash the underlying host server and potentially read data from other virtual machines on the system. So, the problem for public cloud providers, for example, is that the flaw could have enabled an attacker to potentially get access to other resources and data on the cloud. Needless to say, that would have been catastrophic for any public cloud provider, especially the world’s largest.

Ethernet cloud rental network © Brian A Jackson ShutterstockProper precautions

The issue only affects hardware-assisted virtual machines (HVMs) and not paravirtualized (PV) virtual machines. HVMs leverage capabilities within silicon, including Intel’s VT-x and AMD-V.

The flaw was first reported two weeks ago to the open-source Xen Project by SUSE Linux employee Jan Beulich. In contrast with the Heartbleed vulnerability in April and the Shellshock vulnerability that was first reported Sept. 24, the open-source Xen project was able to keep details of the CVE-2014-7188 flaw private until the major public cloud providers could be patched.

The Xen Project has been run as a Linux Foundation Collaboration Project since 2013. Xen has had a detailed security response process in place since 2011 that has been incrementally updated many times to refine the process.

“If a vulnerability is not already public, we would like to notify significant distributors and operators of Xen so that they can prepare patched software in advance,” the Xen security response process document states. “This will help minimise the degree to which there are Xen users who are vulnerable but can’t get patches.”

Software vulnerabilities are an inevitable fact of modern applications. What the Xen project has managed to achieve is a way of properly managing the bug fixing process, without the hype and hysteria that is associated with zero-day bug disclosure. More importantly, by getting all the major cloud providers fixed before the flaw was publicly disclosed, the Xen Project likely saved the IT world from a major security nightmare.

Are you a cloud pro? Try our quiz!

Originally published on eWeek.