Putting Kafka In Jail - Best Practices To Run Kafka On Kubernetes & DC/OS

How Putting Kafka In Jail Actually Frees You

View on Slideshare

If you’ve worked with Kafka–which is a supported technology included in Cloudflow by Lightbend–then you may have discovered that it’s best suited to run on bare metal on dedicated machines, and in statically defined clusters.

But for many of us, this isn’t convenient for utilizing the full scope of our resources, like mixed-use containers with orchestration technologies like DC/OS and/or Kubernetes, and on-premise, hybrid and cloud infrastructures.

So what can you do about it? Maybe it's time to put Kafka in jail.

In this webinar by Sean Glover (@seg1o), Senior Engineer for Cloudflow, we will review leading Kafka implementations on DC/OS and Kubernetes to see how they reliably run Kafka in container orchestrated clusters and reduce the overhead for a number of common operational tasks with standard cluster resource manager features. You will learn specifically about concerns like:

• The need for greater operational knowhow to do common tasks with Kafka in static clusters, such as applying broker configuration updates, upgrading to a new version, and adding or decommissioning brokers.
• The best way to provide resources to stateful technologies while in a mixed-use cluster, noting the importance of disk space as one of Kafka’s most important resource requirements.
• How to address the particular needs of stateful services in a model that natively favors stateless, transient services.

Watch The Full Video (50 Min)

Watch on YouTube

Using Kafka With Cloudflow

If you're using Kafka to support your streaming data applications, then check out how to leverage this technology along with others like Akka, Spark, Flink, and more. They are all included in Cloudflow, so you can get all the support you need from a single vendor. Schedule a demo of Cloudflow with us here:

SCHEDULE A DEMO

Share

Author(s)

Oliver White

Director of Developer Marketing, Lightbend, Inc.

Twitter: @eOliverWhite

Oliver has been helping startups and enterprises tell their technology stories since 2007. As Chief Storyteller at Lightbend, Oliver has dedicated much of his time to creating educational content and building community awareness around Reactive system architecture and tooling. Oliver is a co-founder of Virtual JUG, the creator of the ZeroTurnaround (acquired by Perforce) content brand RebelLabs, and, somewhat unexpectedly, the coiner of the phrase “SMACK Stack”.