Risk in Reinforcement Learning: Nothing Ventured, Nothing Gained

Wednesday, April 20, 2016
6:00 AM to 7:00 AM
SAC 2.120
Free and open to the public

In many sequential decision problems all that we have is a record of historical trajectories. Building dynamic models from these trajectories and ultimately sequential decision policies may result in much uncertainty and bias. In this talk we consider the question of how to create control policies from existing historical data and how to better sample trajectories so that future control policies would be better. This question has been central in reinforcement learning in the last decade if not more, and involves methods from statistics, optimization, and control theory.

We will focus on one the possible remedies to uncertainty in sequential decision problems: using risk measures such as the conditional value-at-risk as the objective to be optimized rather than the ubiquitous expected reward. We consider the complexity and efficiency of evaluating and optimizing risk measures. Our main theme is that considering risk is essential to obtain resilience to model uncertainty and  model mismatch. 

We will then describe two challenging real-world domains that have been studied in our research group in collaboration with experts from industry and academia: diabetes care management in healthcare and asset management in high-voltage transmission grids. For each domain we will describe our efforts to reduce the problem to its bare essentials as a reinforcement learning problem, the algorithms for learning the control policies, and some of the lessons we learned.

x x


Shie Mannor

Shie Mannor

Technion, Israel Institute of Technology

Shie Mannor is a professor of Electrical Engineering at the Technion, Israel Institute of Technology. Shie graduated from the Technion with a PhD in Electrical Engineering in 2002. He was a Fulbright postdoctoral scholar at LIDS at MIT from 2002 to 2004. He was at the Department of Electrical and Computer Engineering at McGill University from July 2004 until August 2010, where he held a Canada Research Chair in Machine Learning. Shie has been with the Department of Electrical Engineering at the Technion since 2008 where he is currently a professor. His research interests include machine learning, planning and control, and networks. 

Shie has published over 70 journal papers and over 130 conference papers and holds 5 patents. His research awards include several best paper awards, the Henri Taub Prize for Academic Excellence, an ERC Starting Grant, an HP Faculty Award and a Horev Fellowship.