# Compress Large Language Models via Collaboration Between Learning and Matrix Approximation
This folder contains the code utilized for the paper: Compress Large Language Models via Collaboration Between Learning and Matrix Approximation

## Requirements
The code was run using Python 3.10.12. The packages used to run the experiments can be found in the `requirements.txt` file, which contains all the dependencies for setting up the virtual environment.

```
pip install -r requirements.txt
```
# How to Run?
The configurations for each experiment can be found under the file `./config_files/prune_configs.py`. Experiments can be run by calling
```
python run_prune.py run_id 
```
where `run_id` is a positive integer that indexes a specific experiment. Typically, we choose to change the configurations in `./config_files/prune_configs.py`directly and use `run_id`=1 .Prior to running, in `run_prune.py`, fill in where to save the results (`results_path`) and where to save the models for checkpointing during pruning (`checkpoint_path`). 

# Codebases Utilized
Our code utilizes and takes inspiration from the codebases found at the following GitHub Repos:
- SparseGPT: https://github.com/IST-DASLab/sparsegpt
- Wanda: https://github.com/locuslab/wanda
- OWL: https://github.com/luuyin/OWL
- OATS: https://github.com/stephenqz/OATS