Abstract: 

Unsupervised learning and generative models have a long history and recent methods have combined the generality of probabilistic reasoning with the scalability of deep learning to develop learning algorithms that have been applied to a wide variety of problems giving state-of-the-art results in image generation, text-to-speech synthesis, and image captioning, amongst many others. Advances in deep generative models are at the forefront of deep learning research because of the promise they offer for allowing data-efficient learning, representation learning and for model-based reinforcement learning and decision making.

In this talk I will cover two recent works on using unsupervised learning and generative models to form useful representations of complex 3D environments and to improve the data efficiency of reinforcement learning (RL).

When humans walk around a complex 3D scene, they are capable of understanding it in terms of the objects that compose it, their properties such as shape, color, position, weight, etc as well as their interactions such as collisions and occlusions. Similarly when agents interact with a complex environment, they must form and maintain a representation and beliefs about the relevant aspects of that environment.

We show that predictive algorithms coupled with expressive generative models can form stable belief-states in visually rich and dynamic 3D environments. More precisely, we show that the representations learned by these algorithms capture important aspects of the environment such as the layout of the map, the position and orientation of the agent, the position and properties of different objects and their relations in a completely unsupervised manner.

Furthermore, our experiments show that reinforcement learning agents that use these representations have substantially higher data-efficiency on a number of RL tasks compared to strong model-free baselines.

Bio: 

Danilo is a Staff Research Scientist at Google DeepMind, where he works on general-purpose probabilistic machine reasoning and learning algorithms. He a BA in Physics and MSc in Theoretical Physics from Ecole Polytechnique (Palaiseau – France) and from the Institute of Theoretical Physics (SP – Brazil) and a Ph.D. in Computational Neuroscience at Ecole Polytechnique Federale de Lausanne, EPFL (Lausanne – Switzerland). His research focus on scalable inference methods, generative models of complex data (such as images and video), applied probability, causal reasoning and unsupervised learning for decision-making.