Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Commit

Permalink
1. Shell and python file to only download image chat data. 2. Updated… (
Browse files Browse the repository at this point in the history
#2381)

* 1. Shell and python file to only download image chat data. 2. Updated readme

* Adding Copyright

* incorporating pylint suggestions

* running autoformat.sh

* related to #2381. rolling back after autoformat

* lint suggestions

* lint suggestions
  • Loading branch information
shubhamagarwal92 committed Feb 7, 2020
1 parent 5e9f9f6 commit 917baab
Show file tree
Hide file tree
Showing 3 changed files with 49 additions and 0 deletions.
1 change: 1 addition & 0 deletions parlai/tasks/image_chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ Tags: #Image_Chat, #All, #Visual, #ChitChat

Notes: If you have already downloaded the images, please specify with the `--yfcc-path` flag, as the image download script takes a very long time to run

If you just want to download data, run as `./parlai/tasks/image_chat/download_data.sh`. Change the required `$DATA_DIR` variable to where you want to save the file. Defaults to `/tmp`. It basically calls the wrapper `parlai/tasks/image_chat/download_data.py`
33 changes: 33 additions & 0 deletions parlai/tasks/image_chat/download_data.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/usr/bin/env python3
# Copyright (c) Facebook, Inc. and its affiliates.
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

import argparse
from parlai.tasks.image_chat.build import build


def parse_args():
"""
Wrapper to parse CLI arguments.
:return: args
"""
parser = argparse.ArgumentParser()
parser.add_argument(
"-dp", "--datapath", default="/tmp", help="Path where to save data."
)

args = parser.parse_args()
# opts is dic in parlai
args = vars(args)

return args


if __name__ == "__main__":
opt = parse_args()
# Only datapath is required by build.
# Using build function to check the version and
# internal hash
build(opt)
15 changes: 15 additions & 0 deletions parlai/tasks/image_chat/download_data.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env bash
# Copyright (c) Facebook, Inc. and its affiliates.
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

export CURRENT_DIR=${PWD}
# Should return the ParlAI clone path
echo "Running code from: " $CURRENT_DIR

export DATA_DIR=/tmp/
mkdir -p $DATA_DIR
echo "Downloading in data root: " $DATA_DIR

PYTHONPATH=. python parlai/tasks/image_chat/download_data.py \
-dp $DATA_DIR

0 comments on commit 917baab

Please sign in to comment.