Apache Kafka is the leading distributed messaging system, and Reactive Streams is an emerging standard for asynchronous stream processing. The reactive-kafka project combining the two technologies was started in December 2014 by Krzysiek (Chris) Ciesielski from SoftwareMill. Just recently the Akka team joined forces with the project and the resulting collaboration led to the latest 0.11 release, which comes with some major updates. You can read everything about the technical details over on the Akka team blog. I was interested in learning a little bit more about the history and had a chance to interview Chris.
Krzysztof is a software engineer, passionate about programming since his early schooldays. Enthusiast of JVM and Scala ecosystems and open source, Scala Times editor, initiator of the reactive-kafka library, and co-leader of the Lublin Software Craftsmen group.
Markus Eisele: “Thanks for taking the time to answer the questions, Chris! Who are you and what is your day job?”
Krzysiek Ciesielski: “I am a software engineer living in Lublin, Poland and working remotely for SoftwareMill. Almost all projects I’ve been involved with since last 3 years are backend systems based on Scala and Akka.”
ME: “How did you come to Akka Streams and what is your background here?”
KC: “I began to use Akka and was impressed by the actor model, which deals with concurrency in elegant and efficient way. When the first versions of Akka Streams started to take shape, I wanted to test the API and quickly found out that it was a pretty robust tool for building pipelines of flowing data, which is a common case in our projects.”
ME: “Why Kafka? There are other message brokers out there like RabbitMQ, ActiveMQ, and Kestrel.”
KC: “Kafka is particularly important for distributed systems. It focuses on decoupling producers from consumers, and partitions allow designing scalable queues in fault-tolerant clusters.”
ME: “What are the future plans for the module?”
KC: “We are looking forward to feedback as the new API gets adopted as this will help us shape the roadmap. There is also room for performance improvements, to get even better throughput.”
ME: “How can the community help bring it further?”
KC: “The community's role is crucial. The best example is how the current version developed when Alexey Romanchuk joined and practically rewrote all of the API and internals. We also have some great community contributions to tests. People keep coming to the akka/reactive-kafka gitter channel with feedback and questions, which help to set priorities.”
ME: “Are you excited about Alpakka and why?”
KC: “Yes, it’s very promising. Modern systems are often about building data flows between external services, there is a high demand for connector libraries. The Alpakka initiative is a really good response to such demand.”
ME: “Can you describe a little bit around SoftwareMill's engagement with Akka and Reactive in more general and why this is important for you?”
KC: “Our engineers often work on systems based on small services talking through asynchronous messaging. The Reactive approach is a natural fit for such architectures. By leveraging the “let it crash” approach -- bulkheads, streams and other patterns -- we can move away from focusing on thread-level concerns into higher levels of abstractions, not to mention all the possibilities of horizontal scaling. Akka and its modules are leading tools for building such services, that’s why we choose them for so many projects and try to contribute to make them even better.”
Or, if you want to start moving faster today, contact us to get started with Akka and Akka Streams with our new Proof of Value (POV) service:
