NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:4529
Title:Provably Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost

The paper develops finite-time convergence to global optimum for actor-critic in LQR problems. Actor-critic algorithms are hard to analyze, but by focusing on LQR the paper manages to obtain fairly strong results. While LQR in some ways is easier than typical (general) MDPs considered in many RL papers, it is an important problem in robotics and control, so novel and solid results like this work would be nice to publish. The main problems raised by reviewers are (1) lack of (even simple) numerical demonstration; (2) some issues with writing that are mentioned in the detailed reviews.