Covering Disruptive Technology Powering Business in The Digital Age

Using this week’s Spark Summit in San Francisco as a backdrop, Microsoft announced the general availability of Spark for Azure HDInsight, the company’s cloud-based big data analytics platform. As its name suggests, Apache Spark is taking the big data world by storm with an open-source data processing engine that prioritizes speed and ease of use.This week, Microsoft said it is more fully embracing Spark by bringing Spark for Azure HDInsight out of beta. Tiffany Wissner, senior director of Microsoft Data Platform Marketing, described the product as “a fully managed Spark service from Hortonworks that has been hardened for the enterprise and made simpler for you to use,” in a June 6 announcement. “You can also rely on the industry’s highest availability service level agreement for Spark at 99.9 percent.”In a bid to speed up deployments, Microsoft has embarked on several product integrations, including working with Hortonworks—the lead commercial sponsor of Hadoop, on which HDInsight is based—to enhance the YARN resource manager. The company also teamed with Cloudera to co-lead Livy, a REST-based Spark service (currently in alpha) that is intended to provide an easy way for applications to interface with Spark setups.”Spark gives you fast big data processing with a general purpose flexible API,” said Anand Iyer, senior product manager at Cloudera, in a statement. “We see a natural tendency among our customers and partners to want to leverage Spark’s capabilities from client applications that can easily interface with Spark, and Livy makes that possible.”

Microsoft announced that the company plans to make R Server for HDInsight, complete with Spark integration, available on the cloud this summer. R is a statistical programming language that is generally favored by data scientists.

(0)(0)