# Revisiting Semi-Supervised Learning in the Era of Foundation Models

__Official implementation of the following work:__

*Anonymous -- Revisiting semi-supervised learning in the era of foundation models.*


## Introduction
This repository contains the official implementation of *V-PET* (**V**FM-**P**EFT **E**nesmble **T**raining). The code is modified from [USB](https://github.com/microsoft/Semi-supervised-learning). Original copyright notice:

```
Copyright (c) 2021 Othneil Drew
```

## Preparation

### Environment

We recommend using [Conda](https://docs.conda.io/en/latest/) to create a `python=3.9` environment. 

```bash
conda create -n vpet python=3.9
conda activate vpet
```

Then install the required packages:

```bash
pip install -r requirements.txt
```

### Dataset & Pretrained Model

Please download our [models](https://drive.google.com/file/d/1RAm3Q2GeME36dRZ267Z4mLpJF1tBYIuT/view?usp=sharing) and 
[data](https://drive.google.com/file/d/1lXi--k9DY3D_hWP9H1qp7lfJ9PAo-MeY/view?usp=sharing). Then, unzip the files and place them in `data` and `pretrain_weight` directories with the following structure:


```
├── data
│   └── vtab
│       ├── clevr_count
│       │   ├── images
│       │   │   ├── 000
│       │   │   ├── 001
│       │   │   └── ...
│       │   ├── labeded_idx
│       │   ├── test.list
│       │   ├── train.list
│       │   ├── trainval.list
│       │   └── val.list
│       ├── diabetic_retinopathy
│       │   └── ...
│       ├── dtd
│       │   └── ...
│       ├── kitti
│       │   └── ...
│       ├── resisc45
│       │   └── ...
│       └── sun397
│           └── ...
├── pretrain_weight
│   ├── vit_base_patch14_reg4_dinov2_lvd142m.bin
│   └── vit_base_patch16_clip_224_openai.bin
└──[other files]
```

## Workflow

Due to *V-PET* is an ensemble method which requires hyperparameter tuning before training, our workflow is implemented in a three-step process:
1. **Train**: Train the model on labeled data.
2. **Tune**: Based on the trained model, tune the hyperparameters on the validation set.
3. **V-PET**: Run *V-PET* on the pseudo-labels generated by the tuned model.

### Train

All the training commands are in `scripts/` folder. To run all the scripts, we can simply run `run_train.sh` in the root directory. Or we can run our intended commands independently. Note that to run *V-PET* without other SSL baselines, we only need to run the commands in `scripts/clip/run_supervised.sh` and `scripts/dinov2/run_supervised.sh` to train the labeled-only models. 

### Tune

After training the models. We can generate hyperparameter tuning metrics by running the following command:

```bash
# This command will generate the list of models that we need to generate hyperparameter tuning informatics. The list will be saved in `eval_list.pkl`.
python eval_gen_list.py

# This command will read `eval_list.pkl` file and generate the metrics for each model in the list. Results will be saved in each log folder.
python eval.py
```

Then, we can collect the metrics with *SQLite* by running the following command:

```bash
python tune_collect.py
```

### V-PET

After generating the hyperparameter tuning metrics, we can run the following command to generate config files for *V-PET*:

```bash
python gen_config_pet.py
```

Then we can run *V-PET* with the generated config files, for example:

```bash
python train.py --c "config/lora/pet-ensemble/dtd/3-shot/clip/config.yaml"
```

Similarly, we can collect the results using *SQLite*:

```bash
python tune_collect.py
```

### Print Results
To read the tuned results from *SQLite*, we can run the following command:

```bash
python tune_print.py
```
