ISL Colloquium
The ISL Colloquium convenes in person again (on Thursdays at 4pm PT in Packard 101) as of Fall 2021. For those unable to attend, talks are also streamed via Zoom. To avoid "Zoom-bombing", we ask attendees to input their email address here https://stanford.zoom.us/meeting/register/tJckfuCurzkvEtKKOBvDCrPv3McapgP6HygJ to receive the Zoom meeting details via email.

← List all talks ...

[CANCELLED] Global Optimality Guarantees for Policy Gradient Methods

Daniel Russo – Professor, Columbia

Thu, 12-Mar-2020 / 4:30pm / Packard 101

Abstract

Policy gradients methods are perhaps the most widely used class of reinforcement learning algorithms. These methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices. Unfortunately, due to the multi-period nature of the objective, policy gradient algorithms face non-convex optimization problems and can get stuck in suboptimal local minima even for extremely simple problems. This talk with discus structural properties – shared by several canonical control problems – that guarantee the policy gradient objective function has no suboptimal stationary points despite being non-convex. Time permitting, I’ll also discuss (1) convergence rates that follow as a consequence of this theory and (2) consequences of this theory for policy gradient performed with highly expressive policy classes.

This talk is based on ongoing joint work with Jalaj Bhandari.

Bio

Daniel Russo is an assistant professor in the decision, risk, and operations division of the Columbia Business School. His research lies at the intersection of statistical machine learning and sequential decision-making, and contributes to the fields of online optimization, reinforcement learning, and sequential design of experiments. He joined after spending a year as an assistant professor at Northwestern’s Kellogg School of Management and one year at Microsoft Research in New England as a postdoctoral researcher. He received his Ph.D. from Stanford University in 2015 and his B.S. from the University of Michigan in 2011.