Dependencies sometimes cannot be found in the current working directory

[nelson-liu]

Describe the bug

It seems like, sometimes, dependencies aren't properly mounted.

Take this (failing) bundle: https://worksheets.codalab.org/bundles/0xfe7a2a4c16594c03890e4da87aad70d4

It runs a python script that expects ./train.json. However, it fails because it cannot find ./train.json:

06/19/2020 12:03:21 {'wv_file': 'glove.840B.300d.txt', 'wv_dim': 300, 'wv_cased': True, 'sort_all': True, 'threads': 32, 'batch_size': 64, 'pre_processing_type': 'fusion', 'train': 'train.json', 'dev': 'dev.json', 'output_path': 'preprocessed_data.pth', 'spacy_version': '1.9.0'}
06/19/2020 12:03:21 start data preparing...
06/19/2020 12:04:06 glove loaded.
Traceback (most recent call last):
  File "documentreader/prepro.py", line 113, in <module>
    train = flatten_json(trn_file, proc_train)
  File "documentreader/prepro.py", line 77, in flatten_json
    with open(file) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'train.json'

However, if I mimic the bundle and rerun it again ( https://worksheets.codalab.org/bundles/0x39b13132359e4f6d9c100c71bdf139cf ) :

Everything works without a hitch.

Not entirely sure how to reproduce this yet, but I'll update this issue if I find out how.

[nelson-liu]

I realize that the worker is running a version that's a bit newer than the currently deployed version on worksheets.codalab.org . I don't think #2295 is deployed, but it's in my local branch (that the worker is using)—might this be causing the issue? cc @teetone

[percyliang]

@nelson-liu Have you noticed any problems lately? If not, maybe close this and re-open if it happens again?

[nelson-liu]

yes, i've seen this again. still unsure how to reproduce, though. i actually opened a duplicate issue at #2723 , so i'll close this for now.