NeurIPS 2020

DynaBERT: Dynamic BERT with Adaptive Width and Depth

Meta Review

Two reviewers gave strong accept ratings, and the third gave a borderline rating (in the discussion). The reviewers noted the strong empirical results, as well as the novelty of the proposed approach, building on existing works which are well discussed. One minor concern is the complexity of the approach (but as noted by one reviewers, existing methods are also somewhat complicated). I agree with the reviewers, and therefore, the paper is accepted.