MLOps: Monitoring and Managing Drift

Abstract: 

As soon as your machine learning model goes into production everything changes.
You now will need to constantly monitor the performance of your model, evaluate whether it is still sufficient and react accordingly.
This, however, can be a challenge when you have no reliable ground truth to recalibrate its performance.
Typically, you will need to fall back to a surrogate metric that you can measure and that is correlated with the performance of your model.
What those metrics can be and how you track and monitor them is the topic of this workshop.

This workshop consists of two parts:
- Part I: Simulate production on an existing machine learning model and detecting drift
- an OpenAPI machine learning service will be provided
- we will use Evidently, Prometheus and Grafana to monitor and detect the drift
- Part II: Interpreting and Analyzing Drift and what to do about it
- when you have detected drift, you will need to interpret what happened and decide what to do about it
- among the steps you can take is to retrain your model with new data
- we might also have to consider to rethink the model architecture or the data we are using

Our objective is to ensure that you are equipped with the essential knowledge and practical tools to proficiently manage
your machine learning models in a real-world production environment.

All services will be provided as Docker images and can be run locally on your machine.
To have the full experience, you should have Docker and Docker Compose installed.
Notebooks will additionally be provided as a Colab service so you can also run them without a local installation as a fallback.

Bio: 

Oliver Zeigermann has been developing software with different approaches and programming languages for more than 3 decades. In the past decade, he has been focusing on Machine Learning and its interactions with humans.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google