Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tailed Noise

Ta Duy Nguyen, Thien H Nguyen, Alina Ene, Huy Nguyen

Advances in Neural Information Processing Systems 36 (NeurIPS 2023) Main Conference Track

In this work, we study the convergence in high probability of clipped gradient methods when the noise distribution has heavy tails, i.e., with bounded $p$th moments, for some $1