Extraction and recovery of spatio-temporal structure in latent dynamics alignment with diffusion models [NeurIPS'2023 Spotlight]

Yule Wang, Zijing Wu, Chengrui Li, and Anqi Wu

Georgia Institute of Technology

Atlanta, GA, USA

                                                         

Oct.19th Update

Adding nn.init.eye_(self.low_d_readin_t.weight) for the linear probing layers for further alignment stability. 

Environment Setup

To install the required dependancies using conda, run:

$ conda create --name erdiff --file requirements.txt

To install the required dependancies using Python virtual environment, run:

$ python3 -m venv erdiff
$ source erdiff/bin/activate
$ python3 -m pip install --upgrade pip
$ python3 -m pip install -e .

Latent Dynamics Visualization

Cited as

@article{wang2024extraction,
  title={Extraction and recovery of spatio-temporal structure in latent dynamics alignment with diffusion model},
  author={Wang, Yule and Wu, Zijing and Li, Chengrui and Wu, Anqi},
  journal={Advances in Neural Information Processing Systems},
  volume={36},
  year={2024}
}

ERDiff Poster for NeurIPS 2023