# Requirements

```
accelerate==0.29.2
appdirs==1.4.4
bitsandbytes==0.43.1
black==24.4.0
black[jupyter]
datasets==2.18.0
fire==0.6.0
transformers==4.40.0
gradio==4.26.0
sentencepiece==0.2.0
protobuf==5.26.1
torch==2.2.2
```

# Dataset

Get the train data and benchmarks from [here](https://github.com/AGI-Edgerunners/LLM-Adapters)

# Usage

After that, modify the `train_aurora.sh` to your own path.

`--model_p_or_n`: the path or name of model

To **reproduce** our results, just use:

```
bash train_aurora.sh
```

# Acknowledge

This code is based on the [MoSLoRA](https://github.com/wutaiqiang/MoSLoRA/tree/main/commonsense_reasoning), we thank for their efforts.
