Skip to content

Latest commit

 

History

History
124 lines (94 loc) · 5.8 KB

README.md

File metadata and controls

124 lines (94 loc) · 5.8 KB

From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval

This is a repository contains the implementation of our SIGIR'23 full paper From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval. network structure

Table of Contents

Environments

  • Ubuntu 20.04
  • CUDA 11.7
  • Python 3.7

Install other required packages by

pip install -r requirements.txt

Datasets

Following the previous work, we conduct experiments on three fashion related datasets, i.e., FashionAI, DARN, and DeepFashion. Please download and put them in the corresponding folders.

Download Data

FashionAI

As the full FashionAI has not been publicly released, we utilize its early version for the FashionAI Global Challenge 2018. You can first sign up and download two training subsets:

  • fashionAI_attributes_train1.zip(6G)
  • fashionAI_attributes_train2.zip(7G).

Once done, you should uncompress and link them into the data/FashionAI directory.

DARN

As some images’ URLs have been broken, only 214,619 images are obtained for our experiments. We provide with a series of URLs for the images. Please download them into a pic directory that should be created in data/DARN directory.

DeepFashion

DeepFashion is a large dataset which consists of four benchmarks for various tasks in the field of clothing including category and attribute prediction which we use for our experiments, in-shop clothes retrieval, fashion landmark detection and consumer-to-shop clothes retrieval. Download the images into a img directory that should be created in data/DeepFashion directory.

Configuration

The behavior of our codes is controlled by configuration files under the config directory.

config
│── FashionAI
│   ├── FashionAI.yaml
│   ├── s1.yaml
│   └── s2.yaml
├── DARN
│   ├── DARN.yaml
│   ├── s1.yaml
│   └── s2.yaml
└── DeepFashion
    ├── DeepFashion.yaml
    ├── s1.yaml
    └── s2.yaml

Each dataset is configured by two types of configuration files. One is <Dataset>.yaml that specifies basic dataset information such as path to the training data and annotation files. The other two set some training options as needed.

If the above data directory is placed at the same level with main.py, no changes are needed to the configuration files. Otherwise, be sure to correctly configure relevant path to the data according to your working environment.

Training

Download Google pre-trained ViT models for our Patch-aware Branch:

wget https://drive.google.com/file/d/1N2rdQcbhegIOB4fHpifi92w1Lp86umN1/view?usp=sharing

RPF is trained in a two-stage way. For the first stage, we need to train the region-aware branch. Run the following script that uses default settings:

python main.py --cfg config/<Dataset>/<Dataset>.yaml config/<Dataset>/s1.yaml

Based on the trained region-aware branch, the second stages jointly train the whole RPF:

python main.py --cfg config/<Dataset>/<Dataset>.yaml config/<Dataset>/s2.yaml --resume runs/<Dataset>_s1/model_best.pth.tar

Evaluation

Run the following script to test on the trained models:

python main.py --cfg config/<Dataset>/<Dataset>.yaml config/<Dataset>/s2.yaml --resume runs/<Dataset>_s2/model_best.pth.tar --test TEST

We release several pretrained models:

Performance

Expected MAP on FashionAI Dataset

skirt length sleeve length coat length pant length collar design lapel design neckline design neck design overall
RPF 66.75 67.86 59.65 73.23 75.72 73.18 74.40 75.01 70.11

Expected MAP on DARN Dataset

clothes category clothes button clothes color clothes length clothes pattern clothes shape collar shape sleeve length sleeve shape overall
RPF 45.18 54.92 55.08 63.51 57.04 63.54 41.20 86.95 62.43 58.80

Expected MAP on DeepFashion Dataset

texture fabric shape part style overall
RPF 15.62 8.30 15.02 7.38 4.77 10.22

Citation

If you find this repository useful, please consider citing our paper:

@inproceedings{RPF2023,
  title={From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval},
  author={Jianfeng Dong and Xiaoman Peng and Zhe Ma and Daizong Liu and Xiaoye Qu and Xun Yang and Jixiang Zhu and Baolong Liu},
  booktitle={Proceedings of the 46rd International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year={2023}
}