Trustworthy Evaluation Frameworks for Natural Language Generation Technologies


In this talk, I propose a framework for trustworthy evaluations for NLP in general with an application to natural language generation in particular. I posit that trustworthiness emerges from trusting the process by which we build technologies as well as trusting the end product as well by devising practical metrics that correlate well with users' perception of performance.


Mona Diab is a Research Scientist with Facebook AI and she is also a full Professor of CS at the George Washington University where she heads the CARE4Lang NLP Lab. Before joining FB, she led the Lex Conversational AI project within Amazon AWS AI. Her interests span building robust technologies for low resource scenarios with a special interest in Arabic technologies, (mis) information propagation, computational socio-pragmatics, NLG evaluation metrics, and resource creation. She has served the community in several capacities: Elected President of SIGLEX and SIGSemitic. She currently serves as the elected VP-Elect for ACL SIGDAT, the board supporting EMNLP conferences. She has delivered tutorials and organized numerous workshops and panels around Arabic processing. She is a cofounder of CADIM (Consortium on Arabic Dialect Modeling, previously known as Columbia University Arabic Dialects Modeling Group), in 2005, which served as a world renowned reference point on Arabic Language Technologies. Moreover, she helped establish two research trends in NLP, namely computational approaches to Code Switching and Semantic Textual Similarity. She is also a founding member of the *SEM conference, one of the top tier conferences in NLP. She currently serves as the senior area chair for multiple top tier conferences. She has published more than 250 peer reviewed articles.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google