# Efficient LASER
Here we provide an early pre-release of our code base (based on the prior release of LASER) as a supplement to our submission.


## How to run sample code

We first discuss installing the code and then discuss how to run an experiment.

### Installation

To run an experiment, please install as shown below. We primarily use pytorch and the datasets and transformers package from huggingface.

```bash
pip3 install -r requirements.txt
```

Should you wish to run with HotPot (which requires LLaMA weights), you will need to update the install as follows:


```bash
pip3 install tokenizers==0.13.3
pip3 install --no-cache-dir transformers==4.31.0 sentencepiece
pip3 install protobuf
```


### Running sample code

At the moment, each setup is its own file. To run an experiment that performs a single Clustering LASER transformer to GPTJ on the Fever dataset, you can run:

```bash
python3 src/intervention_gptj_fever.py --lname fc_in --rate 9.9 --lnum 26 --num_clusters 16
```

If you wish to run the 100 Evalation process across all parameters for GPTJ on the Fever dataset, you can run (which will save a .csv file):
```bash
python3 src/intervention_gptj_fever_run_all.py
```

Note that if you wish to run with CounterFact, you need to specify the data file that the data is located and for HotPot, the location of LLaMA weights.

For the 100 Gradients approach defined in our paper, we do not yet provide generalized functionality for this, but to see an example of GPTJ and Roberta with 16 clusters on TruthfulQA and CounterFact respectively, you can run:
```bash
python3 src/gptj_base_statistics.py
```
```bash
python3 src/roberta_base_statistics.py
```
Should you desire, with appropriate commenting/uncommenting in these files, you may obtain the results of other datasets and clustering levels.

