Abstract: 

Nowadays, most machine learning models in production are trained offline with static data inputs. However, data patterns change over time, which can lead to the distribution shift between train data where models are trained and real-world data where models are applied. The discrepancy arises over time, resulting in degrading predictive power in production models. A drift detection and handling mechanism would be required to maintain and further improve model performance for the continual success of AI models in production.

In the talk, we will be presenting the continual learning strategy we are using to address the drift problem and build a sustainable AI system in production. It focuses on consistently responding to changes in data patterns and proactively reacts to performance decays in production, while collects data intelligently over time and evaluates newly trained models on that data for improved performance. The goal is to keep the model performance consistently competitive via long-term model iterations.

Additionally, we will be covering the facilitated infrastructure and resources required to implement the continual learning, to build a sustainable system that is as automated and adaptive as possible, where human involvement is minimized and resources re-utilization is maximized. It includes a codified AutoML framework, MLOps tools, streamlined model deployment and integration pipeline, holistic model monitoring system, task trigger and scheduler system, to be able to continually handles incoming data, performs model development and deploys the new models in a reliable, sustainable, cost effective and time-saving manner that take us a long way down the road of AI adoption in business.

Bio: 

Ke works as lead data scientist in Data Science & Analytics Lab (DSAL) at American Family Insurance. She leads data science and engineering team to build AI-powered solutions turning business initiatives into large-scale ML models, reusable ML systems and ML reliable products. She is a believer, advocator and practitioner in AI adoption to advance business and society growth. Her expertise is in solving data-driven problems, designing data science strategy, and building scalable end-to-end ML applications in production. Ke earned her Master’s degree in Statistics at University of Illinois Urbana-Champaign in 2018, and B.S. in Statistics at Renmin University of China.