Supplementary Material for Paper #7685
Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function

This folder contains appendix for the paper, as well as the code to plot each figure in the paper (./code).

GDchaos_appendix.pdf is the appendix for the paper, containing some lemmas, proofs and numerical evidence.

About the folder ./code:
All the codes are programmed on MATLAB R2018b.
Sec 3.1 (Invariant distribution): Sec31InvariantDistribution.m
Sec 3.2 (Invariant distribution for non-periodic f_1,eps): Sec32NonPeriodic.m; Sec32Quasiperiodic.m
Sec C.1 (Bifurcation): SecC1PeriodDoubling.m
Sec C.2 (Test for Matyas Function): SecC2ErgodicityMatyas.m; SecC2InvDisMatyas.m; SecC2Matyas.m
Sec C.3 (About Lyapunov Exponent): SecC31LyapExpEps.m; SecC31LyapExpEta.m; SecC32QuasiperiodicLyapExpEps.m; SecC32QuasiperiodicLyapExpEta.m; SecC33LyapExpMatyasEps.m; SecC33LyapExpMatyasEta.m
Sce C.4 (Optimization using Momentum): SecC41HeavyBall.m; SecC42NAG.m