Big data streaming frameworks today must process immense amounts of data from an expanding set of disparate data sources. Beyond the requirements of fault tolerance, scalability and performance, compelling streaming engines should provide a programming model where computations can be easily expressed and deployed in a distributed manner. Akka is a natural fit to meet these requirements.
GearPump is built on top of Akka. It uses an actor tree to model the whole streaming system. With this hierarchy, many features of Akka such as error supervision, isolation and concurrency etc. are leveraged to build a resilient, scalable and performant system.
With 4 nodes, the whole throughput can reach 11 million messages per second (100 bytes per message) with average latency of 17ms.Weihua JiangEngineering Manager for BigData Enabling, Intel
Inspired by this story? Contact us to learn more about what Lightbend can do for your organization.
Share This Case Study
Lightbend is the proud provider of the world's leading Reactive application development platform. We are a passionate crew of technology pioneers committed to building amazing software. We build and maintain the Play web framework, the Akka message driven runtime, and the Scala programming language. Our mission is to help developers build high-performance applications that are responsive resilient, elastic and message driven.