If you’re an oenophile (a connoisseur of wines), then finding a bottle of Chateau Mouton Rothschild from 1945 would make you pretty excited. However, machine learning algorithms don’t exactly get better with age. The fact is that most machine learning algorithms are designed to work with stationary data, and the reality is that streaming data is rarely stationary. What happens over time is called "Concept Drift", where models based on data collected within a fixed time period begins to suffer from a loss of prediction quality.
The most common method to deal with concept drift is periodically retraining the models with new data. The length of the period is usually determined based on the cost of retraining. The changes in the input data and the quality of predictions are not monitored, and the cost of inaccurate predictions is not included in these calculations. A better alternative is monitoring the model quality by testing the inputs and predictions for changes over time, and using change points in retraining decisions. There has been significant development in this area within the last two decades.
In this webinar, Emre Velipasaoglu, Principal Data Scientist at Lightbend, Inc. will review the successful methods of machine learned model quality monitoring.
If you'd like to learn more, visit these Lightbend resources to keep up with this emerging competitive advantage:
As always, Lightbend is here to make your streaming, Fast Data, and Machine Learning journey successful. To find out more about our platform subscription, getting-started engagement services, or anything else, feel free to contact us below and schedule a 20-min introduction.
