NeurIPS 2020

Monotone operator equilibrium networks

Meta Review

This paper presents an extension to earlier work on deep equilibrium models (DEQ) improving them in several key aspects: using an alternative parameterization that leads to better training properties and is easy to implement, providing a thorough analysis of the proposed approach including convergence guarantees (an aspect that was lacking both theoretically and in practice for DEQ). The reviewers all agreed that this is an important topic in general and that the paper makes a significant contribution - both theoretically and in terms of solid experimental results demonstrating the benefits of the approach. The reviewers did discuss about some additional supporting experiments that would be important to include (comparison to DEQ, NODE/ANODE with larger networks). The authors did provide insight wrt. this in the rebuttal that seems sufficient. As a result the paper should be accepted. I would stress that the additional experiments should make it into the final version of the paper, perhaps by simply extending Table 1 and adding a short note on the problems they encountered experimentally with DEQ.