In this two-part blog series, we introduce the concepts and benefits of working with both spark-submit and the Kubernetes Operator for Spark. In Part 1, we introduce both tools and review how to get started monitoring and managing your Spark clusters on Kubernetes. In Part 2, we do a deeper dive into using Kubernetes Operator for Spark.

In the first part of this blog series, we introduced the usage of spark-submit with a Kubernetes backend, and the general ideas behind using the Kubernetes Operator for Spark. In this second part, we are going to take a deep dive in the most useful functionalities of the Operator, including the CLI tools and the webhook feature. At the end, we review the advantages and disadvantages of both spark-submit and Operator.

In a recent survey, 75% of over 2400 developers revealed that they are already using (or planning to use) microservices to enable real-time streaming use cases, showing that a key characteristic of Fast Data architectures is the use of microservices for streaming functionality...

In this webinar with Mike Kelland, VP of Global Services at Lightbend, we will share some details of our specialized enablement strategy that allows teams of all sizes to successfully adopt Fast Data technologies and techniques.

From real-time data streaming expert and VP of Fast Data Engineering at Lightbend, Dean Wampler, Ph.D., comes the 2nd Edition of his popular O'Reilly eBook Fast Data Architectures for Streaming Applications.

Two technologies–Akka Streams and Kafka Streams–have emerged as popular tools to use with Apache Kafka for addressing the shared requirements of availability, scalability, and resilience for both streaming microservices and Fast Data. So which one should you use for specific use cases?

You’re probably heard of the “SMACK Stack”, but you may not know that three Lightbend experts–Sean Glover, Stavros Kontopoulos, and Gerard Maas–that recently teamed up with O’Reilly Media on a free eBook about working with the SMACK stack for Fast Data applications (available in PDF, EPUB, and MOBI formats–or all 3 in a .zip folder)!

Machine Learning is one of the hottest topics in software engineering today. While it can deliver major competitive advantages, it’s also very challenging: creating the models is just the first step... 

In this webinar by Dr. Dean Wampler, VP of Fast Data Architecture at Lightbend, Inc., we cut through the buzz around Fast Data and explore how to successfully exploit this new opportunity for innovation in how your organization leverages data.

In our latest survey, Need For Speed: Fast Data Development Trends, over 2400 developers share their use of batch or streaming for data processing workloads...

In The Need For Speed survey, we asked over 2400 developers what’s hard about Fast Data through the lens of the complete project lifecycle: Design, Build, and Run.

Introducing Lightbend Reactive Architecture - Professional, a crash course in thinking differently, guided by experienced Lightbend expert who provides codified lessons learned, best practices, reference architectures, design patterns and analysis techniques when working with Reactive architectures.

Lightbend is happy to announce the release of version 1.1 of Fast Data Platform. Lightbend Fast Data Platform makes it easy for customers to develop, integrate, and operate a system comprising microservices and streaming data pipelines.

In this fireside chat style presentation, Dean Wampler of Lightbend and Matt Aslett of 451 Research will discuss the business justification for transitioning from batch-oriented big data to stream-oriented fast data, and how some of these requirements can be met by leveraging what organizations already know about microservice architectures.

In this webinar with Lightbend Principal Engineer, David Brinegar, we review how OpsClarity is purpose-built to intelligently monitor streaming applications and help Ops teams resolve issues quickly.

In conjunction with our partner O'Reilly, Lightbend is pleased to be able to offer you this expert guide to Machine Learning.

Back in May 2017, we launched our annual charity survey that looked into the rapidly evolving Fast Data, Streaming, and IoT ecosystem. We're thrilled that we received over 2400 responses—reaching our goal of $5000 donated to charity—as well as the opportunity to deeply examine the responses of thousands of developers regarding fast data adoption trends across three key areas—business value, use case, and technology.

We blew through September, which is a month where even the most southern parts of Germany (where I'm from) send all their boys and girls back to school. This monthly Tech Digest brings you the latest news around Scala, Akka, Lagom and Play. A big thank you to this awesome community for all your contributions!

Back in May 2017, we launched our annual charity survey that looked into the rapidly evolving Fast Data, Streaming, and IoT ecosystem. We're thrilled that we received over 2400 responses, reaching our $5000 donation goal for the Kogics Foundation, a public charitable trust in India run by Lalit Pant that provides free education to underprivileged children (including programming in Scala via Kojo), financial and in-kind help to the poor, and even medical assistance to those in need.

"Autumn is a second spring when every leaf is a flower." That is how Albert Camus put it. We made it back from summer vacation and it is time to gear back up into work mode. Here's a good opportunity to start over is to check in with what you might have missed last month. A big thank you to this awesome community for all your contributions!

This webinar by Emre Velipasaoglu, Principal Data Scientist at Lightbend, is for busy Architects and Managers looking to get a handle on what ML is really all about, the ideal use cases for ML and how getting it right can benefit your streaming and Fast Data application architectures. 

"One benefit of Summer was that each day we had more light to read by." The wonderful quote by American author, Jeannette Walls beautifully expresses what summertime was all about for me when I was in school. Summer break was finally there and there's nothing to worry about for a couple of weeks before we had to start over again. Growing older with less vacations than my daughters have today, I still enjoy this time of the year to catch up on reading. And there's a lot to learn because of our awesome community which never stops contributing.

I am Summer, come to lure you away from your computer ... and we all deserve the upcoming break. But this doesn't stop us or our community to create amazing new content and give you some summer time reading. A big thank you to our community for creating so much amazing content sharing the knowledge on how to use Reactive technologies in the right ways.

By now you have likely heard the news that IBM has made a strategic investment in Lightbend to bring Reactive solutions to IBM Platforms. IBM and Lightbend have a history of collaboration, but this strategic investment and relationship will allow enterprise developers to reap the benefits of both Lightbend’s expertise in Reactive microservices and IBM’s expertise in AI and Machine Learning.

Watch this webinar with Dean Wampler, VP of Fast Data Engineering at Lightbend, Inc., to better equip you and your team with the context and background to make the right decision when it comes to adopting streaming frameworks.

With Scala Days just behind us and summer holidays ahead of us, we're gearing up for hot weather with plenty of new releases, events, and community driven articles...

In this webinar, Lightbend’s Senior Product Director, Craig Blitz reviews five early challenges enterprises face when building streaming and Fast Data systems, what to consider and how Lightbend aims to make these choices more straightforward with our upcoming Fast Data Platform.

Throughout our experiences with hundreds of clients, we’ve seen that our most successful customers have been able to truly accelerate their progress with any new technology if they take the journey with a guide from Lightbend...

It's not very often that you can make a big difference by doing very little...see how to convert 6 minutes into a meaningful donation to charity with latest survey!

With April swooshing by, we're a step closer to the warmer months again. Time to check how the communities have been working on Akka, Lagom, Play and Scala and which surprises you may have missed in April.

This article takes a deeper look at how three traditional, brick-and-mortar style businesses–Norwegian Cruise Lines, Swisscom and UniCredit Group–crossed the chasm into Fast Data with technologies, support and services from Lightbend.

In this interview by O'Reilly Media, Lightbend's Duncan DeVore draws over 10 years of experience building Reactive systems to review the basics of what Fast Data is, how to address of design challenges for architects and developers, and discuss the importance of having a Reactive foundation for the next generation of streaming and Fast Data systems.

In this new white paper, we outline the functions a modern monitoring solution must perform in order to truly benefit from the advantages promised by Fast Data and streaming applications.

Now that summer is approaching, it's a good time to check out the latest developments in the Akka, Scala, Play and Lagom communities. With many conferences last month and even more amazing blog posts, our new tech digest gives you a solid overview on what you may have missed last month.

In this Lightbend webinar, we present what to look for in an effective monitoring solution for streaming and Fast Data applications, including a demo of Lightbend Monitoring using sample scenarios to show how we can help your team not just in production, but also during development to catch performance issues before you even deploy.

Lightbend is in the business of delivering frameworks and tools that drive the Reactive (www.reactivemanifesto.org) mantra forward, and when it came time to define a layer of service deployment that drives high resource utilization, enterprise security features and a vibrant ecosystem of like-minded technologies, we selected DC/OS, both as the primary deployment framework for our new Fast Data Platform and the Lightbend Production Suite.

Lightbend attended the first conferences of the season, including JFokus, DevNexus and a number of user group meetups. Now it's time to look into what else the communities around Scala, Akka, Play and Lagom have been up to and pick your lunch break reads for the coming weeks.

Last year, more than 18,000 people registered for over 25 public and private webinars featuring Lightbend engineers, industry luminaries, industry analysts and even a few Lightbend customers, such as Verizon (case study) and PayPal (case study). This comes out to over a full, 24-hour day of presentations, which can be a bit daunting to dive into. So, to make it easier, we've listed the top 10 most popular webinars that we hosted in 2016, according to our registration numbers. We hope you find this list helpful!

Welcome to the first Tech Digest in 2017. The year has been off to an intense start and I'm happy to present the latest installment of the best resources around the Lightbend communities. 

Our Fast Data team at Lightbend has helped create a lot of different services, webinars, books and reports over the last year, so we made a special "mega-pack" of 7 resources to help you get exactly what you need....

In Enterprise Development Trends 2016, we surveyed 2151 JVM engineers about enterprise thoughts around microservices, fast data, cloud infrastructure and container technology. Among tons of fascinating data, we uncovered three major findings from the data collected...

Recently, we published a case study with our customer Swisscom, the leading mobile service provider in Switzerland. While the case study focuses more on the solution and implementation details of how Swisscom used Scala and Spark to build a fast data streaming platform from the ground up in just 9 months, there is more to the story. To get a fuller picture, we sat down with Francois Garillot, Big Data Scientist at Swisscom...

We are happy to announce the availability of Fast Data Architectures For Streaming Applications, a new O'Reilly report authored by the architect for fast data products at Lightbend, Dean Wampler (@deanwampler). In this report he examines the rise of streaming systems for handling time-sensitive problems —like detecting fraudulent financial activity as it happens. You’ll explore the characteristics of fast data architectures, along with several open source tools for implementing them.

People love Apache Spark. Typesafe is the official support partner with Databricks, Mesosphere and IBM for Apache Spark. If you’re a Fast Data visionary, then you’re looking for a modern solution to enable your streaming data applications. In today’s DevOps world, you’re looking for a more accessible way to work in particular with Spark Jobs. Instead of directly mingling around with the specific Spark installation, you want to have access to a lot of functions via lightweight protocols remotely. This is the gap that Spark Job Server closes.

Our whole company is excited about the results from our Going Reactive 2016 survey. So, I asked the source (and my teammate buddy), Oliver White, to give us an overview.

We recently published a white paper called "Fast Data: Big Data Evolved", by Dean Wampler, our traveling Big [Fast] Data Architect at Typesafe and author of Programming Scala. In it, we review the fundamental shift in recent year from what we call "data at rest" to today's demands for "data in motion". 

Today, it's my personal (me, as a person named Tonya Rae Moore who is typing this for you to read) pleasure to unveil the first ever Typesafe Ask Me Anything (AMA) Podcast feat. the Typesafe experts.

A couple of weeks ago, Typesafe launched the results of a survey in which over 2000 people were asked about the explosive adoption of Apache Spark. In the Slideshare presentation embedded above, you can see a sneak preview of some of the results of Apache Spark: Preparing for the Next Wave of Reactive Big Data, but the full version has a lot more to offer. The Scala community is showing intense interest in Apache Spark as well (according to the report, 88% of Spark users are working in Scala, 44% in Java, 22% in Python). So as resident “Apache Spark guy”, I thought it would be nice to put the popularity of Apache Spark in context, looking at what led us here, how enterprises are reacting, and what the needs of the mid-market really are.