Apache Spark: Preparing for the Next Wave of Reactive Big Data
Most so-called Big Data problems today are actually better described in the context of velocity instead of size. You want ‘Fast Data’. Speed is the problem to solve, not size.”
- Jonas Bonér, CTO, Typesafe
Back in summer of 2014, we launched the results of a survey on Java 8, which shared a lot of information we were looking for, but also contained a small golden nugget of data that we didn’t expect: that out of more than 3000 developers surveyed, a shocking 17% of them reported using Apache Spark in production.
Wait, Apache Spark? Yep. Apache. Spark.
Apache Spark is experiencing remarkable growth in both adoption and awareness. Self described as a “fast and general engine for large-scale data processing”. It goes further than that, achieving the ability to process large data between 10-100x faster than MapReduce and enabling event streaming features too. It’s easier to set up, too, from what we hear.
So we did another survey with 2100+ respondents drilling down into what developers, data scientists, executives and organizations are looking forward to with Apache Spark. You can download the full version of the report for the whole story, but here is a sneak peak into the findings that we discovered.
Snapshot of the demographics we recorded:
- 74% are developers, 8% data scientists, 7% C-level execs
- Top 3 industries represented - Telecoms, Banks, Retail
- Top 3 languages used with Spark - Scala, Java, Python
Snapshot of current experience with Spark:
- 31% are evaluating Spark now
- 20% are planning to use Spark in 2015
- 13% run Spark in production now
Snapshot of how current users are benefiting from Spark:
- 82% of users have Spark to replace MapReduce
- 78% of users need faster processing for large data sets
- 67% of users plan to introduce event stream processing
- 62% of users load data into Spark via HDFS
- 54% of users run Spark standalone
We invite you to get the full report, which has been beautifully designed for your reading, and to share it with your colleagues. If you’re interested in where your Big Data requirements could lie in the future, then this report is right for you.