Putting Kafka In Jail - Best Practices To Run Kafka On Kubernetes & DC/OS

How Putting Kafka In Jail Actually Frees You

If you’ve worked with Kafka–which is a supported technology included in Lightbend Fast Data Platform–then you may have discovered that it’s best suited to run on bare metal on dedicated machines, and in statically defined clusters.

But for many of us, this isn’t convenient for utilizing the full scope of our resources, like mixed-use containers with orchestration technologies like DC/OS and/or Kubernetes, and on-premise, hybrid and cloud infrastructures.

So what can you do about it? Maybe it's time to put Kafka in jail.

In this webinar by Sean Glover (@seg1o), Fast Data Engineer at Lightbend, we will review leading Kafka implementations on DC/OS and Kubernetes to see how they reliably run Kafka in container orchestrated clusters and reduce the overhead for a number of common operational tasks with standard cluster resource manager features. You will learn specifically about concerns like:

• The need for greater operational knowhow to do common tasks with Kafka in static clusters, such as applying broker configuration updates, upgrading to a new version, and adding or decommissioning brokers.
• The best way to provide resources to stateful technologies while in a mixed-use cluster, noting the importance of disk space as one of Kafka’s most important resource requirements.
• How to address the particular needs of stateful services in a model that natively favors stateless, transient services.

Watch The Full Video (50 Min)

Using Kafka With Fast Data Platform

If you're using Kafka to support your streaming, Fast Data applications, then check out how to leverage this technology along with others like Akka, Spark, Flink, and more. They are all included in Lightbend Fast Data Platform, so you can get all the support you need from a single vendor! 

• DEMO (10 Min) - See Fast Data Platform in action by Dean Wampler, VP of Fast Data Engineering at Lightbend, Inc.
• LEARN (40 Min) - Check out our recent webinar "Akka, Spark, or Kafka? How To Pick The Right Streaming Engine For The Job"
• DOWNLOAD (1 Min) - Get the Technical Overview of Fast Data Platform (PDF)

Is it time to have a brief chat with someone at Lightbend? Just let us know here:

LET'S HAVE A CHAT

Share

Author(s)

Oliver White

Chief Storyteller, Lightbend, Inc.

twitter: @TheOTown

Oliver has been helping startups and enterprises tell their technology stories since 2007. As Chief Storyteller at Lightbend, Oliver has dedicated much of his time to creating educational content and building community awareness around Reactive system architecture and tooling. Oliver is a co-founder of Virtual JUG, the creator of the ZeroTurnaround content brand RebelLabs, and, somewhat unexpectedly, the coiner of the phrase “SMACK Stack”.

Comments

View All Posts or Filter By Tag