# offline-RL-IL

create conda environment
```
conda create -n uniq python=3.9
```

activate conda environment
```
conda activate uniq
```

install dependencies
```
conda install nvidia/label/cuda-12.3.2::cuda-toolkit
conda install -c conda-forge cudnn==8.9.7.29
pip install -U "jax[cuda12]"==0.4.28

conda install -c conda-forge mesalib
conda install -c conda-forge glew
conda install -c menpo glfw3

export CPATH=$CONDA_PREFIX/include
pip install patchelf

pip install d4rl
pip install git+https://github.com/aravindr93/mjrl@master#egg=mjrl
pip install -U -r requirements.txt
```

run the example code (others can be found in run.sh): 
```
CUDA_VISIBLE_DEVICES=0 python -u scripts/train_UNIQ.py \
--env_name=halfcheetah-expert-v2 \
--v_update=rkl_loss --alpha=3.0 --actor_temperature=5.0 \
--weight_decay=0.01 \
--bad_name_list=random,medium \
--bad_size_list=5000,5000 \
--mixed_name_list=random,medium,expert \
--mixed_size_list=500000,500000,100000 \
--is_good_list=0,0,1 \
--is_bad_list=1,1,0 \
# --use_wandb # uncomment to use wandb
```

tree structure
```
.
├── scripts
│   ├── train_UNIQ.py
├── sources
│   ├── algos
│   │   ├── UNIQ
│   │   │   ├── actor.py
│   │   │   ├── critic.py
│   │   │   ├── disc.py
│   │   │   ├── algos.py
│   │   ├── BC
│   │   │   ├── actor.py
│   │   │   ├── algos.py
│   │   │   ├── ...
│   │   ├── ...
│   ├── datasets
│   │   ├── d4rl_dataset.py
│   │   ├── mixed_dataset.py
│   ├── networks
│   │   ├── critic.py
│   │   ├── actor.py
│   │   ├── disc.py
│   │   ├── value.py
│   ├── utils
│   │   ├── __init__.py
│   │   ├── common.py
│   │   ├── env_wrapper.py
│   │   ├── evaluation.py
├── run.sh
├── requirements.txt
```

For the Safety-Gym datasets, please download them using the following anonymous link:
[https://drive.google.com/file/d/1hcld8ZCjvmwk94laL5zWwNx1lqjDxxgT/view?usp=sharing](https://drive.google.com/file/d/1hcld8ZCjvmwk94laL5zWwNx1lqjDxxgT/view?usp=sharing)
