Lightbend Blog | Cluster

Chaoran Yu

Senior Engineer

Stavros Kontopoulos

Senior Engineer
apache-spark kubernetes real-time fast-data pipelines kubernetes-operator spark cluster
February 26, 2019

How To Manage And Monitor Apache Spark On Kubernetes - Part 1: Spark-Submit VS Kubernetes Operator

In this two-part blog series, we introduce the concepts and benefits of working with both spark-submit and the Kubernetes Operator for Spark. In Part 1, we introduce both tools and review how to get started monitoring and managing your Spark clusters on Kubernetes. In Part 2, we do a deeper dive into using Kubernetes Operator for Spark.

Read More

Chaoran Yu

Senior Engineer

Stavros Kontopoulos

Senior Engineer
apache-spark kubernetes real-time fast-data pipelines kubernetes-operator spark cluster
February 26, 2019

How To Manage And Monitor Apache Spark On Kubernetes - Part 2: Deep Dive On Kubernetes Operator For Spark

In the first part of this blog series, we introduced the usage of spark-submit with a Kubernetes backend, and the general ideas behind using the Kubernetes Operator for Spark. In this second part, we are going to take a deep dive in the most useful functionalities of the Operator, including the CLI tools and the webhook feature. At the end, we review the advantages and disadvantages of both spark-submit and Operator.

Read More