Concept Drift: Monitoring Model Quality in Streaming ML Applications
Why Your Machine Learning Models Won't Age Like Wine
If you’re an oenophile (a connoisseur of wines), then finding a bottle of Chateau Mouton Rothschild from 1945 would make you pretty excited. However, machine learning algorithms don’t exactly get better with age. The fact is that most machine learning algorithms are designed to work with stationary data, and the reality is that streaming data is rarely stationary. What happens over time is called "Concept Drift", where models based on data collected within a fixed time period begins to suffer from a loss of prediction quality.
The most common method to deal with concept drift is periodically retraining the models with new data. The length of the period is usually determined based on the cost of retraining. The changes in the input data and the quality of predictions are not monitored, and the cost of inaccurate predictions is not included in these calculations. A better alternative is monitoring the model quality by testing the inputs and predictions for changes over time, and using change points in retraining decisions. There has been significant development in this area within the last two decades.
In this webinar, Emre Velipasaoglu, Principal Data Scientist at Lightbend, Inc. will review the successful methods of machine learned model quality monitoring.
Watch The Full Video
Continue Learning With These Resources
If you'd like to learn more, visit these Lightbend resources to keep up with this emerging competitive advantage:
- An Introduction To Fast Data: Get The Free O'Reilly eBook Fast Data Architectures for Streaming Applications
- Machine Learning In Production: Learn more with Serving Machine Learning Models - A Guide to Architecture, Stream Processing Engines, and Frameworks
- Lightbend Fast Data Platform (PDF): Download our Fast Data Platform technical overview to learn more about our easy-on ramp for designing, building, and running streaming and Fast Data applications.
As always, Lightbend is here to make your streaming, Fast Data, and Machine Learning journey successful. To find out more about our platform subscription, getting-started engagement services, or anything else, feel free to contact us below and schedule a 20-min introduction.