How One of the Smart Home Industry’s Hottest Vendors Leverages Akka to Manage Event Streaming for Hundreds of Thousands of Customers
As software continues to eat the world, connected devices and sensors are right at the center of the digital transformation that’s sweeping across every industry. GSM Association predicted recently that by 2020 there will be 24 billion connected devices. Cisco predicted that there would be 37 billion intelligent “things,” and NPD cited a 57 percent growth in home automation products in 2016.
Sensors keep getting cheaper, and interoperability of devices has become standard—but it’s the ability to move data from devices quickly and transform that data into value for users that’s defining the winners in the industries that are being transformed by IoT.
Few understand the importance of Fast Data in IoT as well as Utah-based smart home innovator Vivint Smart Home, which has emerged as an industry leader with 1.2 million customers, $758 million in revenue and a $100 million private equity investment from top tech investor Peter Thiel.
Making the Home Safer and Smarter
With its suite of supported devices, its fleet of more than 2,000 service technicians, and its brilliantly designed mobile app and home console interfaces—Vivint has innovated the smart home way beyond the imaginations of legacy security providers. Fast Company recently named Vivint Smart Home of the World’s 50 Most Innovative Companies of 2017. Vivint Smart Home recently received the Editors’ Choice award from PCMag with this recommendation: “With Vivint Smart Home you get a full-blown home security system that is constantly being monitored, and a comprehensive home automation solution you can control from just about anywhere, including a best-in-class doorbell camera.”
For Vivint, the catalogue of services starts with security, but includes a variety of smart home options, including smart locks, lights, thermostats, garage door openers and cameras. The suite of products integrates with voice assistants Amazon Echo and Google Home for voice control of your entire home. Vivint is not only keeping the home safe, but also helping consumers save time, energy and money. The company is the preferred smart home provider for Airbnb hosts and has partnership with Best Buy to offer an integrated smart home suite in more than 400 stores in 2017.
What Makes it Possible: Fast Data and Learned Models
When we caught up with the Vivint engineering team recently, we learned that the company has created sophisticated methods for applying learned models in real time to drive the actions that its system takes for customers.
Vivint's 1+ million smart home customers each have an average of 20 devices per home, including motion sensors, cameras, thermostats and numerous other devices. That translates to tens of millions of “events” per day being generated by Vivint customers and fed back into the model in near real time. Vivint also gets data from customer interactions with the application, whether on mobile, or from the panel inside the home.
For some events, Vivint doesn’t need to do real-time analysis, as some of its models run daily. But one of the core values of the Vivint platform is the intelligence that it uses to route data, correlate events with opportunities to drive intelligence, and then make decisions that translate to user benefits, such as increased safety, comfort, or cost savings.
Akka at the Center of Real-Time Streaming Events
Akka is a core piece of Vivint’s Fast Data system for handling tens of millions of customer events, sitting between Vivint’s homegrown event bus and its machine learning models. Vivint has a virtual representation of every home, and heavily leverages the Actor model, running tens of millions of Actors on its cluster.
Actors often store the state of events just on the stream—for example, that the battery level on a device has dropped below critical. The state of devices is fed to the models, and each of those is fed to its own cluster shard. When Vivint’s models interpret things such as whether the house is unoccupied (so that a thermostat could be turned down for energy cost savings), they look at triggers from the device Actors, apply the events to the models, and then determine the appropriate output, using Akka to receive messages.
The initial reason Vivint chose Akka was for its concurrency controls. Previously the company used a pure Java application that ran up against thread management challenges. One aspect that contributed to this was the use of many timer-based tasks. Some of the models required a certain interval of time to elapse, after which point, a new action should take place. Even on small beta groups of a few hundred internal users, thread resources would quickly get locked up and required much more low-level management of thread executor pools. The introduction of Akka provided an abstraction from this low-level thread management that in most cases needed minimal tweaks to default configuration to get the best performance.
As Vivint scaled its Fast Data system, the team realized that in addition to more elegant ways to scale concurrency, it needed isolation. In the initial system, all of the processes for a single customer were managed in a very narrow class hierarchy. There were cases where a lower priority task, for example, updating to the most recent version of an occupancy model, would supersede the much higher priority task of being more reactive to customer interactions, such as the application’s responding to current sensor data to make a prediction. Individually, the resultant latency tended to be less noticeable at the customer level, but in aggregate, this sort of sharing of responsibility would lead to multi-second delays before customers could observe their changes. Akka’s paradigm of lightweight, single-purpose actors, following the maxim of “no shared mutable state,” led to an application where one portion of functionality was less capable of negatively impacting another independent portion of functionality.
Because Vivint is using Akka’s cluster sharding, it’s also using Split Brain Resolver, which allows it to make sure there is no duplication of devices and customers. Eliminating the concern for duplication is one key benefit, but isolation and elasticity are another. With not much work, Vivint is able to make the best decisions about distribution of load, about machine resource allocation—setting up one customer panel versus 100,000 panels, for example, is roughly the same, and simplifies testing scenarios. “We already had a message-based architecture, and putting isolation on top of it meant that we’re able to process our data in a highly scalable fashion and quickly,” said Abe Sanderson, Senior Software Engineer at Vivint. “In the last year, we’ve been able to use Akka to service more than 700,000 customers and tens of millions of Actors. Not only does it handle our current load, but we also think the solution could scale 10x without any concurrency or isolation issues.”
Looking Ahead: Advanced ML and AI Use Cases With Akka
Vivint has developed Sky, a cloud-based artificial intelligence that promises to eliminate the need to program your smart home system. You’ll no longer need to write complicated scripts telling your system what it should do in every conceivable scenario or sequence of events, or become frustrated at your inability to predict them all. Sky can accurately detect the state of your home to make intelligent recommendations and take specific actions. For example, Sky uses the data from your sensors and devices to understand whether your home is occupied or vacant. If no one is home, Sky reaches out and asks if you want to lock the doors, close the garage or arm your system.
With Akka at the center of its Fast Data strategy, the company is in a position to capitalize on additional use cases for ML models against real-time streaming data.