Abstract: Reinforcement Learning has seen an explosion of work in the last few years with some high-profile results such as DeepMind’s success at Go with AlphaZero. Most of the success has been demonstrated on games which, while impressive, have certain properties that don’t translate to real-world challenges faced by practitioners. First, these games can be simulated efficiently at the massive scales needed to train the RL algorithms. Second, many of the games, like Chess, Go, and Breakout, are fully observable, in that everything about the state of the world is available to the learning agent at each iteration. For many sequential decision processes, however, there may be no simulator and the state of the world is only partially observable at any given time.
Imitation Learning is a related approach to Reinforcement Learning, but instead of having the AI agent learn from scratch through its own exploration, Imitation Learning is about learning decision policies from expert demonstrations. This then becomes a supervised learning problem that tries to learn the policy of the expert rather than a policy that maximizes long-term reward. While Imitation Learning may not produce superhuman-level performance on competitive tasks like games, it can achieve human-level performance on other tasks, such as controlling a tracking camera during a sports broadcast.
This talk will introduce the formal Imitation Learning problem and discuss two main categories of agent training in the Imitation Learning paradigm: behavioral cloning and interactive experts. The talk will also include examples of where it could be used to solve real-world problems and a demonstration of the DAgger algorithm.
Bio: Byron Galbraith is the Chief Data Scientist and co-founder of Talla, where he works to translate the latest advancements in machine learning and natural language processing to build AI-powered conversational agents. Byron has a PhD in Cognitive and Neural Systems from Boston University and an MS in Bioinformatics from Marquette University. His research expertise includes brain-computer interfaces, neuromorphic robotics, spiking neural networks, high-performance computing, and natural language processing. Byron has also held several software engineering roles including back-end system engineer, full stack web developer, office automation consultant, and game engine developer at companies ranging in size from a two-person startup to a multi-national enterprise.
Byron Galbraith, PhD
Chief Data Scientist | Talla