## Scripts

1. Get the origin model.

   ```bash
   python main_train.py --arch {model name} --dataset {dataset name} --epochs {epochs for training} --lr {learning rate for training} --save_dir {file to save the orgin model}
   ```
   A simple example for ResNet-18 on CIFAR-10.

   ```bash
   python main_train.py --arch resnet18 --dataset cifar10 --lr 0.1 --epochs 182
   ```
2. Generate Saliency Map (only for SalUn)

   ```bash
   python generate_mask.py --save_dir ${saliency_map_path} --model_path ${origin_model_path} --num_indexes_to_replace ${forgetting data amount} --unlearn_epochs 1
   ```
3. Unlearn
   We should integrate our DualOptim into SalUn, SFRon and SCRUB.

   * SalUn

   ```bash
   python main_random.py --unlearn RL --unlearn_epochs ${epochs for unlearning} --unlearn_lr ${learning rate for unlearning} --retain_lr ${learning rate for retaining} --num_indexes_to_replace ${forgetting data amount} --model_path ${origin_model_path} --save_dir ${save_dir} --mask_path ${saliency_map_path} --optim ${sgd, adam, dual...}
   ```
   A simple example for ResNet-18 on CIFAR-10 to unlearn 10% data.

   ```bash
   python main_random.py --unlearn RL --unlearn_epochs 10 --unlearn_lr 0.018 --retain_lr 0.01 --num_indexes_to_replace 4500 --model_path ${origin_model_path} --save_dir ${save_dir} --mask_path mask/with_0.5.pt
   ```
   * SFRon

   ```bash
   python -u main_random_sfron.py --save_dir ${save_dir} --model_path ${origin_model_path} --unlearn SFRon --num_indexes_to_replace ${forgetting data amount} --sfron_alpha ${alpha} --sfron_iters ${epochs for unlearning} --unlearn_lr ${learning rate for forgetting} --retain_lr ${learning rate for retaining} --optim ${sgd, adam, dual...}
   ```
   * SCRUB

   ```bash
   python -u main_forget.py --save_dir ${save_dir} --model_path ${origin_model_path} --unlearn SCRUB --num_indexes_to_replace ${forgetting data amount} --unlearn_epochs 10 --unlearn_lr ${learning rate for forgetting} --retain_lr ${learning rate for retaining}  --optim ${sgd, adam, dual...}
   ```
   The following are the commands for the other baseline methods that should be reproduced with the SFRon-version MIA (different from that used in SalUn):

   * Retrain (already have the results for CIFAR-10, no need to tune the hyperparameters)

   ```bash
   python main_forget.py --save_dir ${save_dir} --model_path ${origin_model_path} --unlearn retrain --num_indexes_to_replace ${forgetting data amount} --unlearn_epochs ${epochs for unlearning} --unlearn_lr ${learning rate for unlearning}
   ```
   * FT

   ```bash
     python main_forget.py --save_dir ${save_dir} --model_path ${origin_model_path} --unlearn FT --num_indexes_to_replace ${forgetting data amount} --unlearn_epochs ${epochs for unlearning} --unlearn_lr ${learning rate for unlearning}
   ```
   * RL (only difference with SalUn is the mask)

   ```bash
   python main_random.py --unlearn RL --unlearn_epochs ${epochs for unlearning} --unlearn_lr ${learning rate for unlearning} --num_indexes_to_replace ${forgetting data amount} --model_path ${origin_model_path} --save_dir ${save_dir}
   ```
   * GA

   ```bash
     python main_forget.py --save_dir ${save_dir} --model_path ${origin_model_path} --unlearn GA --num_indexes_to_replace 4500 --num_indexes_to_replace ${forgetting data amount} --unlearn_epochs ${epochs for unlearning} --unlearn_lr ${learning rate for unlearning}
   ```
   You can find the hyperparameter settings in the SalUn or SFRon paper.

## Evaluation
   1. Use the scripts in the scripts folder to evaluate the results. We run the unlearning process with 5 different seeds.
     We fix retain_lr to 0.01 (SGD) or 1e-4 (Adam) and find the optimal unlearn_lr to get the lowest average gap.
   2. The average gap and std can be calculated by **parse_result.py** where you should input the paths of log files.
