Handling streams of data-especially "live" data whose volume is not predetermined-requires special care in an asynchronous system. The most prominent issue is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination.
Reactive Streams is an initiative to provide a JVM standard for asynchronous stream processing with non-blocking back pressure-which ensures that fast data sources don't overwhelm the stream destination. Since co-founding the Reactive Streams Initiative and releasing Akka Streams 1.0 in July 2015, we've seen interest and adoption continually increase-Reactive and Streams is now part of the upcoming JDK 9 release.
For enterprises dealing with streaming workloads that may never end, Akka Streams has proven to provide the availability, scalability, and resilience needed for production systems. Not only does Akka Streams add the benefits of back pressure and type-safety, it also provides developers with a far greater way to define and visualize their workflows.
With Akka Streams providing the "glue" between both microservices and the real-time analysis performed in the Fast Data space, we wanted to shine the spotlight on three industry leaders in their respective markets–PayPal, Credit Karma, and Flipkart–collectively worth well over $50 billion in market capitalization. Read more to discover the use cases and requirements that led these industry leaders to select Akka Streams, part of the overall Akka toolkit for building massively scalable, highly resilient distributed systems on the JVM.
How does Paypal, one of the world's largest online payment systems, process over 1 billion transactions a day on a fairly minimalistic infrastructure? The power of Akka. PayPal completely rebuilt their legacy Spring web crawler with Akka and Scala after facing the inability to scale and remain resilient with Spring and Java.
PayPal's throughput is back-pressured and managed with Akka Streams' asynchronous processing to handle the workloads that continually increase during processing. By dropping their bloated, thread-based architecture in favor of Akka Streams, PayPal was able to blow past 1 billion transactions per day, while gaining a 10x performance increase for their new Akka-based web crawler.
In 2007, Credit Karma took the world by storm by offering free credit reports, and recently announced an impressive $500M revenue milestone via TechCrunch. So to what does Credit Karma owe their success? Personalization. Credit Karma took the idea of extreme personalization, and crafted out a platform that was able to make personalized recommendations to their members on how to raise their credit score, while additionally saving money. Whereas in the past, Credit Karma was forced to deal with performance issues and out-of-memory errors, today they leverage Akka Streams for reliably handling their massive batch jobs with the added benefits of back pressure to ensure overall system resilience.
With terabytes of data transferred from browsers and mobile devices to their datacenters for processing each day, Akka Streams is a key player in their Fast Data pipeline that ensures the data to flow reliably, without data volumes overwhelming target data stores. In a recent Lightbend webinar, Credit Karma presented their solution that is now capable of handling those terabytes of data seamlessly while making real-time decisions about offerings over 60 million users.
READ THE CREDIT KARMA CASE STUDY
At 100 million users, Flipkart reigns as India's largest e-commerce company. In 2015, as their customer base rapidly increased, scalability and performance issues plagued their platform. Flipkart made the decision to modernize and rebuild their mission critical customer engagement platform with Akka Streams, Akka HTTP, and Apache Kafka. Flipkart's team revealed that Akka streams outperformed their legacy solution by orders of magnitude, all while using far less resources.
The resiliency of Akka allowed Flipkart to create a self-healing, scalable platform that protected them from the risk of expensive downtime. Flipkart increased throughput by 800%, while also reducing their infrastructure footprint by 75%.
Inspired by these success stories? Learn more about how Akka Streams and our other technologies work together in Lightbend Reactive Platform, as well as our upcoming Fast Data Platform. When the time is right for a conversation, you can get in touch with us for a short. 20-min introductory call here:
