Create the environment
```
conda create -n safe -y python=3.7
pip install -r requirements.txt
```

To render a video: 
```
# cd <path>
ffmpeg -y -r 5 -i fig-%d00.png -vf fps=25 -pix_fmt yuv420p -vcodec mpeg4 -vb 2M out.mp4
```

default ckpt: `~/logs/safe/225/ckpt-10000.pt`

`~/logs/safe/367`: real safe zone for 225


# How to add a new env
1. train an init policy: `python run/mf_td3.py --param env/invpen`
2. save the policy ckpt to `configs/env/invpen.json5`
3. train a model: ` python run/main.py --param env/invpen learned pretrain_model`
4. train an init L: `python run/main.py --param env/invpen learned obj/19 MALA pretrain_L`
5. train the safe policy

`ln -s ~/ptml/safe/ckpt ckpt`
