Mind Your Evaluation: Towards Practical Off-Policy Evaluation in Safety Critical Settings

Abstract: 

In high-stakes decision-making domains such as healthcare and self-driving cars, off-policy evaluation (OPE) can help practitioners understand the performance of a new policy before deployment by using observational data. However, when dealing with problems involving large and combinatorial action spaces, existing OPE estimators often suffer from substantial bias and/or variance. In this talk, I will cover several new and practical tools for improving evaluation in safety-critical settings that improve statistical guarantees of estimates, as well as provide more insights on how to perform robust evaluation in situations where traditional assumptions cannot be met. I will draw connections with topics from interpretability, causal inference and uncertainty estimation and discuss how these are all key for evaluation.

Bio: 

Sonali is an Assistant Professor and leader of the AI for Actionable Impact Group at Imperial College London. Her research focuses on decision-making in uncertainty, causal inference and building interpretable models to improve clinical care and deepen our understanding of human health, with applications in areas such as HIV and critical care. Prior to this, Sonali was a postdoctoral research fellow at Harvard. Her work has been published at a number of machine learning conferences (NeurIPS, AAAI, ICML, AISTATS) and medical journals (Nature Medicine, Nature Communications, AMIA, PLoS One, JAIDS). She was also a Swiss National Science Fellow and was named a Rising Star in AI in 2021. Sonali received her PhD (summa cum laude) in 2019 from the University of Basel, Switzerland, where she built intelligent models for understanding the interplay between host and virus in the fight against HIV. Apart from her research, Sonali is also passionate about encouraging more discussion about the role of ethics in developing machine learning technologies to improve society.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google