Opening the Pod Bay Doors: Building Intelligent Agents That Can Interpret, Generate and Learn from Natural Language
Opening the Pod Bay Doors: Building Intelligent Agents That Can Interpret, Generate and Learn from Natural Language


Thanks to advances in imitation and reinforcement learning techniques, we can now train intelligent agents to accomplish a diverse range of goals. But if we want to create household robots or personal assistants that can take advantage of this diversity, we need to give users some way to tell them what to do! This tutorial will focus on humans' favorite tools for communicating goals and plans: natural language. We'll assume basic familiarity with supervised learning and RL, and begin with a review of core machine learning techniques useful for natural language instruction following problems. The body of the talk will focus on modeling techniques for instruction following problems in different kinds of environments and data conditions. We'll conclude with a survey of other applications for the tools we've built, including instruction generation, interpretability, and machine teaching.


Jacob Andreas is an assistant professor at MIT and a senior researcher at Microsoft Semantic Machines. His research focuses on language learning as a window into reasoning, planning and perception, and on more general machine learning problems involving compositionality and modularity. Jacob earned his Ph.D. from UC Berkeley, his M.Phil. from Cambridge (where he studied as a Churchill scholar) and his B.S. from Columbia. He has been the recipient of an NSF graduate fellowship, a Facebook fellowship, and paper awards at NAACL and ICML.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google