Abstract: 

Although powerful, modern machine learning models can be sensitive. Seemingly subtle changes in a data distribution can destroy the performance of otherwise state-of-the art models, which can be especially problematic when ML models are deployed in production.. In this workshop, we will give a hands-on overview to drift detection, the discipline focused on detecting such changes. We will start by building an understanding of the ways in which drift can occur, and why it pays to detect it. We’ll then explore the anatomy of a drift detector, and learn how they can be used to detect drift in a principled manner.

You will work through a real-world example using Alibi Detect, an open-source Python library offering powerful algorithms for adversarial, outlier and drift detection.You’ll learn how to set-up drift detectors, and deduce what type of drift is occurring. Since data can take many forms, such as image, text or tabular data, you’ll explore how to use existing ML models to preprocess your data into a form suitable for drift detectors. Then, to gain further insights into the causes of drift, you’ll employ state-of-the art detectors which are able to perform fine-grained attribution to instances and features. To assess whether model performance has been affected by drift, you’ll experiment with using supervised and uncertainty based detectors. Finally, since in production environments data often arrives sequentially, we’ll discuss the use of online drift detectors that act on single data instances as they arrive.

This hands-on workshop is targeted at a beginner-intermediate level. No prior knowledge or understanding of drift detection is required (we’ll be covering that) but a basic knowledge of machine learning and some experience with Python will be helpful.

Bio: 

Ed Shee, Head of Developer Relations at Seldon. Having previously led a tech team at IBM, Ed comes from a cloud computing background and is a strong believer in making deployments as easy as possible for developers. With an education in computational modelling and an enthusiasm for machine learning, Ed has blended his work in ML and cloud native computing together to cement himself firmly in the emerging field of MLOps.