Benjamin Recht: Optimization Perspectives on Learning to Control (ICML 2018 tutorial)

Abstract:
Given the dramatic successes in machine learning over the past half decade, there has been a resurgence of interest in applying learning techniques to continuous control problems in robotics, self-driving cars, and unmanned aerial vehicles. Though such applications appear to be straightforward generalizations of reinforcement learning, it remains unclear which machine learning tools are best equipped to handle decision making, planning, and actuation in highly uncertain dynamic environments.
This tutorial will survey the foundations required to build machine learning systems that reliably act upon the physical world. The primary technical focus will be on numerical optimization tools at the interface of statistical learning and dynamical systems. We will investigate how to learn models of dynamical systems, how to use data to achieve objectives in a timely fashion, how to balance model specification and system controllability, and how to safely acquire new information to improve performance. We will close by listing several exciting open problems that must be solved before we can build robust, reliable learning systems that interact with an uncertain environment.
Presented by Benjamin Recht (UC Berkeley)

[@yviruss1]

Special thanks to the uploader. What a gem of a talk! The link on Brecht's webpage is not 720p (it is 360p), so this is a lifesaver.

[@fenglongsong4760]

Really clear! Best tutorial to talk about the relationship between optimal control and reinforcement learning I've ever seen!

[@kvasios]

Great upload! Thanks!

[@amr.sharaf]

This tutorial is great, thanks for sharing, I have a question, there is an implicit assumption that the system dynamics is non-stochastic, except for the zero-mean noise disturbance e, does these observations / results about sample complexity hold for stochastic MDP dynamics as well?

[@yviruss1]

Not quite. I am assuming you are referring to slide number 42/80 titled discrete MDPs. The sample complexity is modified by the number of iterations, which is not one for the case that you point out.

[@blanamaxima]

sweet talk

[@yviruss1]

On 67/80, sounds something like receding horizon control.