Part of the code is modified from Github repo vilbert-multi-task.
- Download the repository from Github
git clone git@github.com:NoOneUST/COMP5212-Project-GivBERT.git
cd COMP5212-Project-GivBERT
- Install the requirements
pip install -r requirements.txt
- Install pytorch, please check your CUDA version
If you want to run GivBERT, we recommend cuda 10.2 and
pip install pytorch==1.5 torchvision==0.6
If you want to run VilBERT, use
conda install pytorch==1.4 torchvision cudatoolkit=10.1 -c pytorch
To setup the data, you can either download the data provided by vilbert-multi-task, or download from Google Drive which is a pruned-version especially for this project.
TBC
To get move on, you need to download the pre-trained VilBERT models for 12-in-1: Multi-Task Vision and Language Representation Learning. Please put the models under model folder. The download links are listed below:
cd ./GivBERT
cd ./
python main.py --bert_model bert-base-uncased --from_pretrained model/<model_name> --config_file config/bert_base_6layer_6conect.json --lr_scheduler 'warmup_linear' --train_iter_gap 4 --save_name <finetune_from_multi_task_model>