SAN FRANCISCO, CA--(Marketwired - Feb 23, 2016) - Lightbend, the company behind the Reactive Platform that includes Play Framework, Akka and Scala, today announced a collaboration with Intel to bring low-latency and high availability to Big Data streaming pipelines running on the Java Virtual Machine (JVM). As "Fast Data" becomes a requirement for every enterprise seeking to make use of its data assets for analytics and real-time decision making, Intel is utilizing Akka and the Reactive Streams specification from Lightbend to accelerate innovations in data streaming.
According to a November 2015 Gartner report authored by W. Roy Schulte (Add Event Processing to Your Business Analytics Repertoire), "Unfortunately, many organizations discard or ignore some of this data because no one knows what to use it for and no one has built applications to process it. For another large percentage of the data, organizations store and analyze it days or weeks later using data discovery and advanced analytics tools. However, relatively little of this data is processed in real time or near real time when it is freshest and potentially most valuable, at least for certain purposes. This is despite the fact that most business managers clearly understand the benefits of situation awareness and fast response to changing conditions. Managers typically don't have access to systems that would process event streams in near real time because their companies don't include stream processing as part of their normal business analytics or application development practices."
Lightbend and Intel are collaborating to allow organizations to embrace "data in motion" as systems evolve from traditional batch architectures to stream-based architectures. Specifications such as Reactive Streams, and stream processing libraries such as Akka Streams and Spark Streaming, provide the standards and technologies necessary to implement such systems. Lightbend delivers the leading Reactive application development platform for the JVM, enabling enterprises to build massively distributed applications at a fraction of the time, with unparalleled uptime and cost savings.
GearPump is a lightweight real-time big data streaming engine. It is inspired by recent advances in the Akka framework and a desire to improve on existing streaming frameworks like Storm, Millwheel and Spark Streaming. The name GearPump is a reference to the engineering term "gear pump", which is a simple pump design that consists of only two gears, but is very powerful at streaming water.
Akka (recent winner of the JAX Innovation Award for "Most Innovative Open Source Tech in 2015") is a toolkit for building message-driven applications. With Akka Streams, Akka has incorporated a graphical domain specific language (DSL) for composing data streams, an execution model that decouples the stream's staged computation from its execution (allowing for actor-based, single-threaded and fully distributed and clustered execution), type safe stream composition, an implementation of the Reactive Streaming specification that enables back-pressure, and more than 20 predefined stream "processing stages" that provide common streaming transformations that developers can tap into (for splitting streams, transforming streams, merging streams, and more).
Intel and Lightbend will explore how to make GearPump compatible with the Reactive Streams API and to optimize Akka for running on Intel hardware platforms. This includes optimization for SSD and persistent memory.
GearPump is a supported project within the Trusted Analytics Platform (TAP) open source effort, founded by Intel. Instead of starting from scratch and deploying a host of different tools, packages and services, TAP provides everything needed by data scientists and developers to create applications powered by Big Data Analytics. The extensible environment combines many open-source components into a single, integrated platform in which capabilities are exposed as easy-to-integrate tools and services rather than having to recreate existing features. This allows data scientists and developers to focus on their areas of expertise and value rather than getting mired in a complex integration project.
Additional Materials
