Abstract: 

A.I. has a trust issue. Part of it has to do because models aren't appropriately de-risked but also another part has to do with communication. It is essential to communicate the model's strengths and weaknesses to stakeholders to manage expectations. More importantly, weaknesses include potential threats, trends, constraints, and uncertainties ranging from fairness to consistency concerns. Unfortunately, the standard predictive performance metrics don't capture all the nuance in these concepts, and they are usually tough to communicate to non-technical stakeholders.

In this session, we will leverage Microsoft's Error Analysis tool to explore different ways of assessing model errors using a traffic prediction problem.

*I use (https://erroranalysis.ai/) and Interpret ML (https://interpret.ml/) tools in my job with similar problems in agriculture (which I cannot disclose). I also use IBM's Uncertainty Quantification 360, which is very useful for the understanding of model uncertainty.

Session Outline:

Lesson 1: XAI and Model Error Analysis and Uncertainty
Short presentation about the importance of Error analysis and uncertainty estimation and what they have to do with XAI
Lesson 2: Hands-On Error analysis Workflow
Leveraging Microsoft's Error Analysis tool to figure out what are the weaknesses of the model.
Lesson 3: Hands-On Uncertainty estimation Workflow
Using IBM's Uncertainty Quantification 360 to understand what inputs create the most uncertainty in the model

Background Knowledge:

Intermediate skills with Python and at least basic understanding of machine learning

Bio: 

Serg Masís has been at the confluence of the internet, application development, and analytics for the last two decades. Currently, he's a Climate and Agronomic Data Scientist at Syngenta, a leading agribusiness company with a mission to improve global food security. Before that role, he co-founded a search engine startup, incubated by Harvard Innovation Labs, that combined the power of cloud computing and machine learning with principles in decision-making science to expose users to new places and events efficiently. Whether it pertains to leisure activities, plant diseases, or customer lifetime value, Serg is passionate about providing the often-missing link between data and decision-making. He wrote the bestselling book "Interpretable Machine Learning with Python" and is currently working on a new book titled "DIY AI" for Addison-Wesley for a broader audience of curious developers, makers, and hackers.