### Multi-agent Dynamic Algorithm Configuration

#### Installation using Anaconda

```bash
conda create -n madac python=3.7.13
conda activate madac
pip install -r requirements.txt # EPyMARL's dependency
pip install -r requirements_moea.txt
```

#### Train MA-DAC

Train MA-DAC in different tasks.

```bash
# Train MA-DAC (3)
python src/main.py --config=vdn_ns --env-config=moea with env_args.key=M_2_46_3
# Train MA-DAC (5)
python src/main.py --config=vdn_ns --env-config=moea with env_args.key=M_2_46_5
# Train MA-DAC (7)
python src/main.py --config=vdn_ns --env-config=moea with env_args.key=M_2_46_7
# Train MA-DAC (M)
python src/main.py --config=vdn_ns --env-config=moea with env_args.key=M_2_46_357
```

Train MA-DAC without the $i$-th agent. (Add `env_args.ban_agent = x`, $x\in\{0,1,2,3\}$). For example,

```bash
python src/main.py --config=vdn_ns --env-config=moea with env_args.key=M_2_46_357 env_args.ban_agent=0
```

Train MA-DAC using specific reward type. (Add `env_args.reward_type = x`, $x\in\{0,1,2,3\}$, default value is 0). For example,

```bash
python src/main.py --config=vdn_ns --env-config=moea with env_args.key=M_2_46_357 env_args.reward_type=1
```

You can modify the relevant configuration file `src/config/envs/moea.yaml` and `src/config/algs/vdn_ns.yaml`

#### Test MA-DAC

The trained model is saved in directory `results/models/`, you need to specify the model directory in the configuration file via parameter `checkpoint_path`. More details of the configuration file can be found in [EPyMARL](https://github.com/uoe-agents/epymarl).

Test MADAC in a specific problem. (The problem set is `DTLZ2_3 DTLZ4_3 WFG4_3 WFG5_3 WFG6_3 WFG7_3 WFG8_3 WFG9_3 DTLZ2_5 DTLZ4_5 WFG4_5 WFG5_5 WFG6_5 WFG7_5 WFG8_5 WFG9_5 DTLZ2_7 DTLZ4_7 WFG4_7 WFG5_7 WFG6_7 WFG7_7 WFG8_7 WFG9_7   `) For example,

```bash
python src/main.py --config=vdn_ns_test --env-config=moea_test with env_args.key=DTLZ2_3 
```

Test MA-DAC without the $i$-th agent. (Add `env_args.ban_agent = x`, $x\in\{0,1,2,3\}$). For example,

```bash
python src/main.py --config=vdn_ns_test --env-config=moea_test with env_args.key=DTLZ2_3 env_args.ban_agent=0
```

#### Other Baselines

##### MOEA/D

```bash
python src/envs/moea/moead_baseline.py --baseline # MOEA/D
python src/envs/moea/moead_baseline.py --action 1 0 1 0 # MOEA/D-OP1
python src/envs/moea/moead_baseline.py --action 1 1 1 0 # MOEA/D-OP2
python src/envs/moea/moead_baseline.py --action 1 2 1 0 # MOEA/D-OP3
python src/envs/moea/moead_baseline.py --action 1 3 1 0 # MOEA/D-OP4
```

##### MOEA/D-FRRMAB

```bash
python src/envs/moea/frrmab.py
```

##### MOEA/D-AWA

```bash
python src/envs/moea/awa.py 			# MOEA/D-OP2-AWA
python src/envs/moea/awa.py --baseline 	# MOEA/D-AWA
```

##### DQN

Train DQN in different tasks. (The task set is `M_2_46_3, M_2_46_5, M_2_46_7, M_2_46_357`). For example, 

```
python src/envs/moea/dqn.py --key M_2_46_3
```

The trained model can be found in the directory `src/envs/moea/model/dqn`

The command to test the corresponding model on all problems is

```
python src/envs/moea/test_dqn.py --key M_2_46_3
```

##### MA-UCB

```bash
python src/envs/moea/maucb.py
```

#### License

All the source code that has been taken from the `EPyMARL` repository was licensed (and remains so) under the Apache License v2.0 (included in `LICENSE` file). Any new code is also licensed under the Apache License v2.0.

