# Self Forcing

## Installation
(Optional) Create a conda environment
```
wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
bash ~/miniconda.sh -b -u -p ~/miniconda3
source ~/miniconda3/bin/activate ""
conda create -n self_forcing python=3.10 -y
conda activate self_forcing
```

Install dependencies
```
pip install -r requirements.txt
pip install flash-attn --no-build-isolation
python setup.py develop
```

## Download checkpoints and text prompts
```
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B --local-dir-use-symlinks False --local-dir wan_models/Wan2.1-T2V-1.3B
huggingface-cli download self-forcing/Self-Forcing checkpoints/ode_init.pt --local-dir .
huggingface-cli download self-forcing/Self-Forcing checkpoints/self_forcing_dmd.pt --local-dir .
huggingface-cli download self-forcing/Self-Forcing checkpoints/self_forcing_gan.pt --local-dir .
huggingface-cli download self-forcing/Self-Forcing checkpoints/self_forcing_sid.pt --local-dir .
huggingface-cli download self-forcing/Self-Forcing vidprom_filtered_extended.txt --local-dir prompts
```

## Inference
DMD:
```
python inference.py \
    --config_path configs/self_forcing_dmd.yaml \
    --output_folder videos/self_forcing_dmd \
    --checkpoint_path checkpoints/self_forcing_dmd.pt \
    --data_path prompts/MovieGenVideoBench_extended.txt \
    --use_ema
```

SiD:
```
python inference.py \
    --config_path configs/self_forcing_sid.yaml \
    --output_folder videos/self_forcing_sid \
    --checkpoint_path checkpoints/self_forcing_sid.pt \
    --data_path prompts/MovieGenVideoBench_extended.txt \
    --use_ema
```

GAN:
```
python inference.py \
    --config_path configs/self_forcing_gan.yaml \
    --output_folder videos/self_forcing_gan \
    --checkpoint_path checkpoints/self_forcing_gan.pt \
    --data_path prompts/MovieGenVideoBench_extended.txt \
    --use_ema
```

## Training
Note: We provide training scripts for DMD/SiD that are data-free. Since GAN training requires generating a large synthetic dataset, uploading the full set would have been time-prohibitive. The original training code relied on internal libraries, so we have migrated it to a version that no longer contains any internal dependencies, to the best of our ability. However, we have not yet fully verified that this updated code produces results identical to the original. Training was performed on 8 nodes, each with 8 GPUs of 80 GB memory.

DMD:
```
torchrun --nnodes=8 --nproc_per_node=8 --rdzv_id=5235 \
  --rdzv_backend=c10d \
  --rdzv_endpoint $MASTER_ADDR \
  train.py \
  --config_path configs/self_forcing_dmd.yaml \
  --logdir logs/self_forcing_dmd \
  --disable-wandb
```

SiD:
```
torchrun --nnodes=8 --nproc_per_node=8 --rdzv_id=5235 \
  --rdzv_backend=c10d \
  --rdzv_endpoint $MASTER_ADDR \
  train.py \
  --config_path configs/self_forcing_sid.yaml \
  --logdir logs/self_forcing_sid \
  --disable-wandb
```