NeurIPS 2020

Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework

Meta Review

This paper proposes a unified framework for solving inverse RL, system identification, and optimal control problems, using implicit differentiation through Pontryagin’s Maximum Principle (PMP). The paper applies this framework to imitation learning, system identification, and optimal control tasks in four experimental domains, and shows improved performance compared to other methods. According to R1, who has a more traditional control theory background, in their effort to present this unified view of these problems, the authors make the implicit claim that the method is equally useful in all three modes: system identification, optimal control, and inverse optimal control. In reality, the proposed method presents an inductive bias from the PMP that is most useful for the first and third modes (SysID and IOC), where the current dominant approach is the inverse KKT method from Toussaint and others and bespoke SysID methods. In terms of optimal control, the proposed unification seems more of an abstraction and interpretation, but does not offer a new method. That said, it is necessary to include it in the paper because it is a building block for the other modes. Reviewers said that this paper is novel and exciting enough to be accepted, and I definitely agree. I am not aware of other methods that differentiate through the PMP.