This paper studies the mode connectivity phenomenon of neural network and tries improve mode connectivity using a technique called neuron alignment. The paper demonstrated (both theoretically and empirically) that neuron alignment can improve mode connectivity. While the reviewers all find the paper to be interesting, there were also some concerns that the theoretical result is focused on an upperbound of the loss and does not prove that the actual expected loss improves.