Microsoft Brings Hadoop Big Data To Windows Azure

Microsoft has launch launched its Azure cloud-based Hadoop service HDInsight, explaining the move is part of its plan to bring big data to one billion people, and has suggested it signals a shift similar to that which made data-processing tools such as Excel commonplace.

“Microsoft’s perspective is that embracing the new value of data will lead to a major transformation as significant as when line of business applications matured to the point where they touched everyone inside an organisation,” said Quentin Clark, corporate vice president of Microsoft’s data platform group, ahead of a Tuesday keynote at Strata + Hadoop World 2013 in New York City.

‘Major transformation’

HDInsight, offered on Microsoft’s Windows Azure cloud platform, includes the Apache Hadoop data processing platform along with its associated tools. The offering uses Hortonworks’ flagship Hadoop distribution, Hortonworks Data Platform (HDP).

Part of Microsoft’s strategy to popularise big data is to integrate HDInsight with Microsoft’s own business intelligence tools, including Excel, SQL Server and PowerBI, Clark said. As part of that integration, Microsoft worked with Hortonworks on HDP, contributing tens of thousands of lines of code that are to be included back into the open source Apache Hadoop code base, Clark said.

Hortonworks made HDP 2.0 generally available last week, and HDP 2.0 for Windows Server will become generally available next month, according to Clark. He said Hadoop v2 will be supported by HDInsight in “a future update”, but no definite timetable has yet been announced.

Big Data vision

Microsoft’s vision for big data is similar to the its original mission of putting a computer on every desktop, but the company has recognised this approach is in need of updating in the post-PC era. Clark said the big data “transformation” will occur “when anyone with a question that can be answered by data, gets their answer”.

Integration with Microsoft’s BI tools means that, for instance, a user can deploy an Excel feature called Power BI to draw data from Hadoop MapReduce, which can then be automatically analysed and visualised.

The version of Hadoop used by HDInsight will be fully compatible with HDP, allowing workloads to be easily shifted to non-Azure deployments, and Microsoft is also billing HDP for Windows Server as offering an easy migration path to the cloud-based HDInsight.

Microsoft has been testing HDInsight in full production mode with several clients for “a number of months”, according to Clark, with customers including the City of Barcelona, which uses it to analyse the effectiveness of public services, and Virginia Tech, which uses it to offer access to DNA sequencing tools and resources.

Amazon Web Services also offers Hadoop, while Rackspace and IBM have both said they will offer cloud-based Hadoop distributions soon.

Do you know all about IT in the movies? Take our quiz!

Matthew Broersma

Matt Broersma is a long standing tech freelance, who has worked for Ziff-Davis, ZDnet and other leading publications

Recent Posts

US Approves SpaceX Starlink For Planes, Trains And … Ships

US FCC regulator gives its official approval for SpaceX to use its Starlink satellite internet…

5 hours ago

Bitcoin Falls Below $19,000, But Recovers Slightly Friday

Ominous sign for crypto markets? The value of Bitcoin dropped over 6 percent to below…

6 hours ago

Meta Slashes Hiring As It Braces For Downturn – Report

CEO Mark Zuckerberg tells staff to brace for a deep economic downturn, as Meta cuts…

7 hours ago

Silicon In Focus Podcast: Connected Business

Is the definition of a ‘connected business’ very different today than it was just two…

9 hours ago

BT Disappointed As CWU Votes To Strike, Despite 5 To 8 Percent Pay Rise

First strike in 35 years after BT staff with the e Communications Workers Union vote…

1 day ago