!pip install transformers
https://pytorch.org/get-started/locally/
!pip install pandas
!pip install numpy
!pip install matplotlib
!pip install torchdiffeq
!pip install torchmetrics
!pip install torchviz
!pip install torch-fidelity
- CIFAR-10
- Publicly available at: https://www.cs.toronto.edu/~kriz/cifar.html
- For the Caption Generation check
Caption_Generation.ipynb
-
Caption_Generation.ipynb:
- Utilizes the BLIP2 model to generate descriptive captions for images in the CIFAR dataset and stores the resulting dataset as a pickle file.
-
Cross_Validation.ipynb:
- Implements code for cross-validation using a list of learning rates.
-
Flow_Matching_Training.ipynb:
- Encompasses the entire training process, employing flow matching with a conditional optimal transport objective in conjunction with the proposed UNet model.
-
Flow_Inference.ipynb:
- Contains code for generating images from uniformly sampled inputs and evaluates the FID and CLIPScore metrics for the trained models.
-
Text_Encoding.ipynb:
- Utilizes the BLIP2 tokenizer to convert captions into tokens for subsequent use in the conditioning process.
-
UNet_Attn.ipynb:
- Houses the proposed UNet model, a key component in the conditional and unconditional image generation tasks.
-
Docs
- Project Report: Contains the documented project with the Problem Statement, Data Augmentation, Methodology, UNet Model, and the Results
-
Dependency Installation:
- Execute the command to install project dependencies necessary for proper functioning.
-
Repository Cloning:
- Clone the project repository to the local machine using the command:
git clone https://github.com/Anshumaan-Chauhan02/Guided-Flow-Matching
- Clone the project repository to the local machine using the command:
-
Caption Generation:
- Run the
Caption_Generation.ipynb
notebook to generate a captioned dataset utilizing the BLIP2 model.
- Run the
-
Flow Matching Training:
- Execute the
Flow_Matching_Training.ipynb
notebook to initiate the training process for the unconditional/conditional generation model.
- Execute the
-
Model Evaluation and Inference:
- Run the
Flow_Inference.ipynb
notebook for comprehensive model evaluation and generation of inferences.
- Run the
Note:
- Ensure to update the specified file paths in the notebooks with the appropriate local repository path.