Machine learning systems in production are subject to performance degradations due to many external factors and it is vital to actively monitor system stability and integrity. A common source of model degradation is due the inherent non-stationarity of the real world environment, commonly referred to as data drift. In this presentation, I will describe how to reliably quantify data drift in a variety of different data paradigms including Tabular data, Computer Vision data, and NLP data. Attendees of this talk will come away with a conceptual toolkit for thinking about data stability monitoring in their own models, with example use cases in common settings as well as in more challenging regimes.
Drift Detection in Structured and Unstructured Data
Keegan Hines, PhD
