In tarfiles.tarfile_in_dropbox, we don't get credentials for the pnfs dropbox case

[shreyb]

Reproduce the issue:

$ jobsub_submit -G gm2 --use-pnfs-dropbox  --tar-file-name tardir:///home/sbhat/FileUploadTests -n --verbose 3 file:///usr/bin/sleep 300
./
./testfile2
./testfile3
./tar-dash-f-mix.sh
./dash-f-dropbox.sh
./testfiles.tar
./tar-file-name.sh
./testfile
./dash-f.sh
gfal-mkdir error: 13 (Permission denied) - HTTP 401 : Authentication Error
Traceback (most recent call last):
  File "/opt/jobsub_lite/bin/jobsub_submit", line 233, in <module>
    main()
  File "/opt/jobsub_lite/lib/tracing.py", line 161, in wrapper
    res = func(*args, **kwargs)
  File "/opt/jobsub_lite/bin/jobsub_submit", line 153, in main
    do_tarballs(args)
  File "/opt/jobsub_lite/lib/tracing.py", line 161, in wrapper
    res = func(*args, **kwargs)
  File "/opt/jobsub_lite/lib/tarfiles.py", line 266, in do_tarballs
    path = tarfile_in_dropbox(args, tfn[8:])
  File "/opt/jobsub_lite/lib/tarfiles.py", line 364, in tarfile_in_dropbox
    fake_ifdh.mkdir_p(os.path.dirname(location))
  File "/opt/jobsub_lite/lib/fake_ifdh.py", line 276, in mkdir_p
    raise PermissionError(f"Error: Unable to make directory {dest}")
PermissionError: Error: Unable to make directory https://fndcadoor.fnal.gov:2880/GM2/resilient/jobsub_stage/6fbfd6cc7a960f82dca01aa4dadcef2e03819a7d4a387664c49db71a0bedabb1

This happens because in tarfiles.tarfile_in_dropbox - https://github.com/fermitools/jobsub_lite/blob/master/lib/tarfiles.py#L301 - we only get the credentials in the CVMFS case (like this line: https://github.com/fermitools/jobsub_lite/blob/master/lib/tarfiles.py#L316). In the PNFS case (--use-pnfs-dropbox), we don't do the same, so job submission will fail due to inability to authenticate.

I believe this isn't an issue once a vault token exists on the machine, though. But either way, I think we need to put a get_creds call in the PNFS steps here.