fix!: Validate paths for Job Bundles

[marofke]

What was the problem/requirement? (What/Why)

We need to safeguard users when uploading inputs. There's a few cases here:

1. A Job bundle directory cannot contain symlinks that resolve outside of the bundle directory
2. The default PATH values in the template defined in the bundle:
   a. cannot have default parameters that are absolute, and
   b. the default relative paths must resolve inside the bundle directory (so no escaping with ../)
3. For input paths in the Asset References file, any PATH values in the Job bundle parameters file, as well as any queue environment PATH parameters:
   a. If the path resolves outside of the job bundle directory AND is not part of a configured storage profile location for the queue being submitted to, we need to warn the user, so they're aware files will be uploaded from an unexpected area

What was the solution? (How)

1. When loading a job bundle, walk the directory and ensure all paths resolve inside the resolved job bundle directory (as the Job Bundle directory could itself be a symlink)
   a. This was mostly added in src/deadline/client/job_bundle/parameters.py and src/deadline/client/job_bundle/loader.py
2. Reorganize the submission workflow, adding a new function to src/deadline/job_attachments/upload.py to first process all asset paths (these are combined with the validated Job Bundle, parameters file, queue environment parameters, and asset references files), and prompt the user to confirm if they want to continue submission if files are found outside of the Job Bundle dir and any storage profile locations for the queue
   a. Note that I moved the confirmation popup that happens AFTER hashing to BEFORE hashing to include a combined confirmation message
   b. Also note: when submitting through the GUI, since we move the job bundle to the ~/.deadline/job_history/ dir, the actual input files for the job bundle do not move, so if the files in the original job bundle are not part of a Storage Profile filesystem location, it will always prompt the user
3. I also took the opportunity to clean up our GUI a little bit (I haven't done much QT work before, so it was fun to try paying the beauty tax)
   a. Fixed some text that didn't render as it was using HTML tags
   b. Grouped the progress bars so it's more clear

What is the impact of this change?

Users should have more information about what they're submitting when they attempt to submit a Job Bundle. This protects users from job bundles that could contain symlinks that resolve outside of the job bundle directory. Also, the UI is a little nicer to look at (at least I think so ;) )

How was this change tested?

• unit / integration tests pass
• LOOOOTS of manual submission testing (for both CLI and GUI workflows):
  • Job bundle with symlinks that resolve within the job bundle -> submits successfully
  • Job bundle with symlinks that resolve outside the job bundle -> raises an error
  • Job bundle with relative default PATH values that resolve outside the bundle dir -> raises an error
  • Job bundle with absolute default PATH -> raises an error
  • Set a storage profile filesystem location, included asset references within that directory -> normal submission with no warnings
  • Set a storage profile filesystem location, included asset references outside that directory -> makes a warning popup
  • Added a queue environment with IN and INOUT PATH parameters. Confirmed that absolute paths that resolved outside of my configured storage profile locations presented a warning popup, and relative / absolute paths that resolved within storage profiles did not

UI Changes

Old UI

• The status boxes at the bottom had cut off text and white borders that didn't fit with the overall theme
  
• Submission progress UI didn't have information clearly separated
  

New UI

• Made the credential stats boxes (at the bottom) look more in line with the rest of the styling
  
  
  
  
• Made the submission progress UI more clear
  
• Warning when files are outside of SP file location / job bundle dir
  
• What happens if you hit cancel
  
• Error when you try to load a Job Bundle with paths that resolve outside of the bundle dir
  
• What the submission UI looks like if you have all valid paths within a SP file location / bundle dir
  
• Successful submission UI
  
  

Was this change documented?

Not yet

Is this a breaking change?

Yes, the interface for the Job Attachments S3AssetManager has changed (should only affect custom Python scripts that import the library)

• hash_assets_and_create_manifest takes different arguments
• prepare_paths_for_upload was added and needs to be called before hash_assets_and_create_manifest

Also:

• the user is prompted for confirmation before hashing when submitting (previously, they'd be prompted after hashing, before uploading)
• can no longer open job bundles with paths that resolve outside of the job bundle directory
• can no longer open job bundles withe default PATH parameters in the template, and/or relative paths that resolve outside of the job bundle directory

[marofke]

I've also updated the screenshots with some UI changes I made based on Mark's feedback, take another look at the description!

[gahyusuh]

Thanks for making lots of refactoring and making the GUI submitter prettier! I posted minor comments mainly related to variable names.

[gahyusuh]

Look great to me, thank you!