## SOLAR: Synergy-Oriented LeARning

## Setup

```bash
pip install -r requirements.txt
```

## Running

```
python main.py 
--custom_xml environments/humanoids 
--actor_type transformer 
--critic_type transformer 
--grad_clipping_value 0.1 
--attention_layers 3 
--attention_heads 2 
--lr 0.0001 
--transformer_norm 1 
--attention_hidden_size 256 
--condition_decoder 1
--label SOLAR
--save_buffer 1
--save_freq 100000
--start_timesteps 2000
```

We were using [Sacred](https://github.com/IDSIA/sacred) and [WandB](https://wandb.ai) for experiment management. 
You can disable WandB in `modular-rl/src/main.py` by setting `enable_wandb=False`.  

To train other environments, change the `--custom_xml` to `hoppers`, `walkers`, 
`walker_humanoids_hopper`, `unimalA`, `unimalB`, `unimalC`.

## Acknowledgement

- The code is built on top of [Amorpheus](https://github.com/yobibyte/amorpheus) and [SMP](https://github.com/huangwl18/modular-rl) repository.
- Initial implementation of the transformers was taken from the [official Pytorch tutorial](https://pytorch.org/tutorials/beginner/transformer_tutorial.html) and modified thereafter. 
