Skip to content

Latest commit

 

History

History
95 lines (86 loc) · 2.76 KB

DATA_PREP_README.md

File metadata and controls

95 lines (86 loc) · 2.76 KB
Raw

Dataset Preparation

Note that the following steps are required only if you want to prepare the annotations from parent repository. If you just want to run the model with the annotations see DATA_README.md

Here I have outlined the steps to prepare the following datasets:

  • Flickr30k Entities
  • ReferIt
  • Unseen splits

We convert the annotations for each dataset into .csv file with the format: img_id, bbox, queries

The project directory is $ROOT

Flickr30k Entities

Current directory is located at $FLICKR=/some_path/flickr30k

  1. To get the Flickr30k Images you need to fill a form whose instructions can be found here http://shannon.cs.illinois.edu/DenotationGraph/. Un-tar the file and save it under $FLICKR/flickr30k_images
  2. git clone https://github.com/BryanPlummer/flickr30k_entities.git. Unzip the annotations. At this point the directory should look like:
$FLICKR
|-- flickr30k_entities
    |-- Annotations
    |-- Sentences
    |-- test.txt
    |-- train.txt
    |-- val.txt
|-- flickr30k_images
|-- results.json
  1. Make a symbolic link to $FLICKR here using ln -s $FLICKR $ROOT/data/flickr30k
  2. Now we prepare the flickr30k entities dataset using
cd $ROOT
python data/prepare_flickr30k.py
  1. The above code does the following:
    • Convert the annotations in .xml to a single .json file (because it is easier to deal with dictionaries and better to read only once). It is saved in $FLICKR/all_ann.json
    • Create train/val/test .csv files under $FLICKR/csvs/flickr_normal/{train/val/test}.csv
  2. At this point the directory structure should look like this:
$FLICKR
|-- all_ann_2.json
|-- all_annot_new.json
|-- csv_dir
    |-- train.csv
    |-- val.csv
    |-- test.csv
|-- flickr30k_entities
    |-- Annotations
    |-- Sentences
    |-- test.txt
    |-- train.txt
    |-- val.txt
|-- flickr30k_images

ReferIt (Refclef)

Current directory is located at $REF=/some_path/referit

  1. Follow the download links at https://github.com/lichengunc/refer to setup referit (refclef). Your folder structure after downloading the images (image subset of imageclef) and the annotations should look like this:
$REF
|-- images
    |-- saiapr_tc12_images
|-- refclef
    |-- instances.json
    |-- refs(berkeley).p
    |-- refs(unc).p
  1. We use only the berkeley split to be consistent with previous works.
  2. Now we again convert to csv format. First create a symbolic link, and then run prepare_referit.py
cd $ROOT/data
ln -s $REF referit
cd $ROOT
ptyhon data/prepare_referit.py

The final structure looks like

$REF
|-- images
    |-- saiapr_tc12_images
|-- refclef
    |-- instances.json
    |-- refs(berkeley).p
    |-- refs(unc).p
|-- csv_dir
    |-- train.csv
    |-- val.csv
    |-- test.csv

Unseen Splits

(Coming soon!)