Microsoft today announced what it called “an extensive commitment” to Apache Spark, an engine for large-scale data processing, bringing several offerings out of preview mode and into general release.
It’s the latest move by Microsoft to work with open-source technologies, part of a broader trend under CEO Satya Nadella. The Redmond tech giant made the announcements in conjunction with Spark Summit 2016, a three-day developers’ conference in San Francisco.
Spark for Azure HDInsight, which runs the Spark open source data analytics framework in the cloud, is now generally available, Microsoft said. R Server for HDInsight, set for general availability this summer, eases moving code and projects to the cloud, without the need for specialized big-data teams, the company said.
R Server for Hadoop is set for general availability in June. It supports both Microsoft’s implementation of the R statistical programming language and native Spark execution frameworks, Microsoft said. The company released a new, free Microsoft R client that allows analyzing data on both local workstations and remote instances of Microsoft R Server. Power BI, a set of data visualization tools, now supports Spark Streaming, now accepts direct input from Spark Streaming, Microsoft said.