Abstract: 

As the world rapidly changing around us, business across industries are treading carefully with unprecedented challenges and, if lucky enough, new opportunities. Due to their statistical assumption of generalizable patterns from the past, machine learning models are facing more scepticism about their validity in the world we now live in. It is more crucial than ever for data scientists to keep close eye on our beloved models in production, understand the impact of business changes on them, and steer promptly from potential pitfalls.

In this session, I will share some experience of model monitoring and diagnosis from a leading UK fintech company. We will discuss how to detect distributional change and analyse its impact on the model performance metrics. We will also look at how to decompose exogenous effect from vintage effect, which would help business understand the model validity in the long run. Finally, we will share some techniques to detect change of statistical relationship and discover new features.

Bio: 

Dr. Jiahang Zhong is the leader of the data science team at Zopa, one of the UK’s earliest fintech company. He has broad experience of data science projects in credit risk, operational optimization and marketing, with keen interests in machine learning, optimization algorithms and big data technologies. Prior to Zopa, he worked as a PhD and Postdoctoral researcher on the Large Hadron Collider Project at CERN, with a focus on data analysis, statistics and distributed computing.