FineRewards

This is the codebase for our paper Boosting Text-to-Image Diffusion Models with Fine-Grained Semantic Rewards

Please refer to the three repositories in the "Acknowledgements" section for environment setup. All experiments are conducted using the FineRewards-RAFT.ipynb notebook.

Caption Reward can be calculated using the "get_cap_reward" function.

SAM Reward can be calculated using the "get_sam_reward" function.

Acknowledgements

IDEA-Research/Grounded-Segment-Anything

lm-sys/FastChat

OptimalScale/LMFlow

Citation

If you find our code helpful, please cite our paper:

@misc{fang2023boosting,
      title={Boosting Text-to-Image Diffusion Models with Fine-Grained Semantic Rewards}, 
      author={Guian Fang and Zutao Jiang and Jianhua Han and Guangsong Lu and Hang Xu and Xiaodan Liang},
      year={2023},
      eprint={2305.19599},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}