At Airbnb, internal tools are just as important as user-facing products, especially where understanding and analyzing user behavior is concerned.

In the past, its analytics team used Cron to manage workflows, but they quickly realized sleep statements were not enough for managing complex dependency hierarchies. Learn how Airbnb front-end and back-end engineering came together to build a distributed system for scheduling data pipelines by using Scala with Mesos.

Our analytics team depends on very complex data pipelines for analysis, necessitating having a Data Infrastructure team for building systems and tools to support this function. We built an in-house called Chronos, which is a distributed system for scheduling data pipelines, which is easier to manage and debug.

Watch the community video


Inspired by this story? Contact us to learn more about what Lightbend can do for your organization.