Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

1. Shell and python file to only download image chat data. 2. Updated… #2381

Merged
merged 8 commits into from
Feb 7, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions parlai/tasks/image_chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ Tags: #Image_Chat, #All, #Visual, #ChitChat

Notes: If you have already downloaded the images, please specify with the `--yfcc-path` flag, as the image download script takes a very long time to run

If you just want to download data, run as `./parlai/tasks/image_chat/download_data.sh`. Change the required `$DATA_DIR` variable to where you want to save the file. Defaults to `/tmp`. It basically calls the wrapper `parlai/tasks/image_chat/download_data.py`
26 changes: 26 additions & 0 deletions parlai/tasks/image_chat/download_data.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env python3

from parlai.tasks.image_chat.build import build
import argparse


def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument(
"-dp", "--datapath", default="/tmp",
help="Path where to save data."
)

args = parser.parse_args()
# opts is dic in parlai
args = vars(args)

return args


if __name__ == "__main__":
opt = parse_args()
# Only datapath is required by build.
# Using build function to check the version and
# internal hash
build(opt)
19 changes: 19 additions & 0 deletions parlai/tasks/image_chat/download_data.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env bash

export CURRENT_DIR=${PWD}
# Should ideally give the parlai clone path
echo "Running code from: " $CURRENT_DIR

#export TASK_DIR="$(dirname "$CURRENT_DIR")"
shubhamagarwal92 marked this conversation as resolved.
Show resolved Hide resolved
#export PARLAI_CODE_DIR="$(dirname "$TASK_DIR")"
#export PROJECT_DIR="$(dirname "$PARLAI_CODE_DIR")"
# Going to the project directory
#cd $PROJECT_DIR

export DATA_DIR=/tmp/
mkdir -p $DATA_DIR

echo "Downloading in data root: " $DATA_DIR

PYTHONPATH=. python parlai/tasks/image_chat/download_data.py \
-dp $DATA_DIR