Code for the submission “Limitations of the Empirical Fisher Approximation”

To run the code, you will have to install the packages provided in the libs folder (move to libs/eftk and libs/efplt and run pip install .)

The code depends on the following packages; numpy, scipy, matplotlib, tqdm and scikit-learn, all available through pip install [x].

The code assume that the datasets (the content of datasets.zip) is stored in a data folder in your home, ~/data.

The main file to run the experiment is experiments/main.py.
Calling python experiments/main.py -h should output the following help:

usage: main.py [-h] (-misspec | -optim | -vecfield) [-run] [-plot] [-appendix]
               [-save] [-show]

Experiment Runner

optional arguments:
  -h, --help  show this help message and exit

Experiment selection:
  -misspec    Misspecification experiment
  -optim      Optimization experiment
  -vecfield   Vector field visualization

Action selection:
  At least one of [-run, -plot] is required

  -run        Runs the experiment and save results as a .pk file
  -plot       Plots the result from a .pk file (requires -save and/or -show)
  -appendix   Also run/plot the experiments in the appendix

Plotting options:
  At least one of [-save, -show] is required if plotting

  -save       Save the plots
  -show       Show the plots

• The -vecfield experiment (Fig. 1) needs to run the various optimizers from the 4 starting points and save the trajectories.

  First call python main.py -vecfield -run, which will save the results in a vecfield.pk file in the current directory.

  You will then be able to call python main.py -vecfield -plot -show.

  The -run part of the experiment should be relatively quick (~10min)

• The -misspec experiment (Fig. 2) does not need pre-computation and the plot can be generated using python main.py -misspec -plot -show

• The -optim experiment (Fig. 3) first needs to do a (rather large) grid search, run with python main.py -optim -run, which will save a optim.pk file in the current directory.

  The plots can then be produced with python main.py -optim -plot -show.

  The grid search can take up to 2 hours depending on the computer used.

  The results in the appendix can be generated (first -run and then -plot) by adding the -appendix flag.

The figures are generated using Matplotlib's Latex backend, and thus need a working latex installation. On Ubuntu, you might need to install apt-get install dvipng