NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:5940
Title:Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization


		
The paper has three interesting contributions: Theoretical contributions: The paper improved the upper bound over the number of communication rounds for non-convex optimization problems under Polyak-Lojasiewicz conditions. While the extension from strong convexity to PL is somewhat expected, the paper is clearly written and handles nicely the "relaxing bounded gradients" assumption. Algorithmic contributions: An adaptive scheme for choosing the communication frequency has been developed. One reviewer raised that the adaptive scheme should have been developed with more details, and that could have been an even more significant contribution (new algorithm). Empirical contributions: Experimental results on Amazon EC2 cluster and an internal GPUs cluster is definitely a plus