# Subfolder structures


We want to study the following problem:

__For the problem of OOD Detection, can we use a SDE model trained with node classification task, possibly with backward diffusion and/or a EBM to enhance OOD Detection?__

In particular, does a SDE classifier by itself provide good metrics for OOD Detection, and is there a connection between the SDE based model and an EBM? 

Recent work like DGEBM (https://arxiv.org/abs/2502.17912) argues that a separated graph encoder model + an additional EBM model can help with OOD Detection task,
can we do better with just a SDE, and for the classifier network, can we regularize it somehow just using a EBM loss? 

What is the connection between the density of the latent state (output of the GNN encoder or SDE encoder) and the OOD score? 

Is there a need to go _backward_ for a SDE model? 

We want to perform a density estimation on the original dataset using the following components:
1. A Graph SDE model vs. a GNN model / Graph encoder
2. A classifier to generate node classification predicted labels. 

with the following variants:
1. Graph SDE vs. GNN 
2. classifier vs. energy-based model using the latent state (trained using score matching vs contrastive divergence)
3. OOD detection usingGraph SDE state variance vs. classifier-induced log-density vs. backward equation-induced log-density. (energy is reciprocal)

For the hybrid models (GNN + EBM) or (SDE + EBM), we also try the "recurrent/iterative optimization scheme" in DGEBM (https://arxiv.org/abs/2502.17912)



# Model Variants 

- Graph SDE based model has 6 variants:
- GNN based models have 2 variants: 
- in total we need to study 8 variants. 

```
GraphSDE + (classifier OR energy-based model) + (state variance OR classifier-induced log-density OR backward equation-induced log-density)

GNN + (classifier OR energy-based model) + (classifier-induced log-density)
```

The latest work by 


## Model Variants 1 (GNN + classifier)

Included in `01-gnn_classifier.ipynb` :heavy_check_mark:


## Model Variants 2 (GNN + energy-based model)

Included in `02-gnn_energy_based.ipynb` :heavy_check_mark:
- also in `02-gnn_energy_based_hyperparams.ipynb` plots the impact of ebm_weight on the behavior. 
- also in `02-gnn_energy_based_iterative.ipynb` plots the impact of ebm only vs. classifier only vs. iterative training. 

## Model Variants 3,4 (SDE + classifier)

Included in `03-sde_classifier.ipynb`. :heavy_check_mark:

- better separation of IND and OOD densities 

> sign shift issue 

We report the metrics on OOD Detection evaluation using:
- classifier-induced energy
- expected predictive entropy
- expected latent space variance

## Model Variants 5,6 (SDE + energy-based model) :heavy_check_mark:

Included in `04-sde_energy_based.ipynb`; this contains 2 variants: 
1. SDE + energy-based model + state variance
2. SDE + energy-based model + backward equation-induced log-density

to save time, also add `04-sde_energy_dsm.ipynb` and `04-sde_energy_ssm.ipynb`. :heavy_check_mark:

> EBM already accounted for by the SDE model + CV loss? 

- `04-sde_energy_iterative.ipynb` plots the impact of ebm only and then added a case of iterative optimization. 


## Model Variant 7 (SDE + classifier + backward equation-induced log-density)
Included in `05-sde_backward.ipynb`

> backward on the first epoch 

## Model Variant 8 (SDE + energy-based model + backward equation-induced log-density)
Included in `06-sde_energy_backward.ipynb`

> SPDE with stochastic matrix 

> unifying uncertainty and OOD detection 


## 01-results_1 

This folder contains a "reverse case", where the model is able to assign low energy to OOD samples and high energy to ID samples.
classifier reached 100% accuracy on supervisation task but this pattern still exists. 

## 02-result

This folder contains a "reverse case", where the model is able to assign low energy to OOD samples and high energy to ID samples. 
Moreover, there exists a training stage where the energy value explodes. 

## 03-result 

This folder conatains the result of GNN classifier that incorporates EBM based loss.


## Observations 

GNN by itself doesn't necessarily calibrate good energy distributions. Moreover, the GNN safe based method doesn't necessarily improve the energy calibration if the OOD samples are present but are not labeled. 


__Challenge (a possible research direction)__: Under a GNN classifier, the SGLD samples generated from the model are not part of the original graph. How to compute the energy gradient for these samples?  (this is because the EBM from a GNN classifier requires both input and edge_index, but the SGLD samples don't have edge_index)





