Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tailed Noise

Part of Advances in Neural Information Processing Systems 36 (NeurIPS 2023) Main Conference Track

Bibtex Paper Supplemental


Ta Duy Nguyen, Thien H Nguyen, Alina Ene, Huy Nguyen


In this work, we study the convergence in high probability of clipped gradient methods when the noise distribution has heavy tails, i.e., with bounded $p$th moments, for some $1