NeurIPS 2020

Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks

Meta Review

The paper studies generalization properties of SGD in non-convex problems by modeling its trajectory with a SDE. To bound the generalization error of the method, the authors consider the Haussdorff dimension of the trajectory of the algorithm. The paper is interesting and technically nontrivial. The paper is timely, as there is a recent interest in heavy-tailed properties of neural network models, and the paper is well written. Author feedback and discussion phase clarified several questions.