Part of Advances in Neural Information Processing Systems 35 (NeurIPS 2022) Main Conference Track
Osbert Bastani, Jason Yecheng Ma, Estelle Shen, Wanqiao Xu
In safety-critical applications of reinforcement learning such as healthcare and robotics, it is often desirable to optimize risk-sensitive objectives that account for tail outcomes rather than expected reward. We prove the first regret bounds for reinforcement learning under a general class of risk-sensitive objectives including the popular CVaR objective. Our theory is based on a novel characterization of the CVaR objective as well as a novel optimistic MDP construction.