fix!: always prompt user about misconfigured inputs

[marofke]

What was the problem/requirement? (What/Why)

We need to better handle some job attachments paths (inputs, outputs, references), specifically around empty/non-existent and misconfigured (i.e. files marked as directories and vice-versa)

What was the solution? (How)

Validate paths better. If any inputs are specified that do not exist, we add them to referenced paths, or fail submission if the new option require_paths_exist is used (either through the CLI flag --require-paths-exist or a checkbox in the Job Attachments tab). Empty input directories are likewise added as referenced paths. Throw an error for any paths that were marked as files but are actually directories.

What is the impact of this change?

Users submitting jobs can be confident all of their expected inputs have been uploaded/included, and have more insight into what is actually being included in their submissions.

How was this change tested?

hatch run fmt
hatch run lint
hatch build
hatch run test

Setup

• I crafted two different kinds of job bundles which I used for various job submissions
  1. Defined IN, INOUT, and OUT parameters, added asset_references and parameter_values that referenced files that did and did not exist, and some directories that were empty
  2. Defined OUT parameters, and added asset_references for input directories that were empty and some that did not exist

Test 1: Submitting Within A Configured Storage Profile Location

1. (GUI) Successful Submission With Confirmation Popup

2. (GUI) Unsuccessful Submission when setting 'Require Paths Exist' checkbox

3. (CLI) Successful Submission With Confirmation Prompt

marofke@ Desktop % deadline bundle submit test_bundle_1    
Submitting to Queue: CMF
INFO:deadline.client.api._submit_job_bundle:Input path './other_input_dir' does not exist. Adding to referenced paths.
INFO:deadline.client.api._submit_job_bundle:Input path '/Users/marofke/Desktop/asset_ref_dir/does_not_exist' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path './other.txt' resolving to '/Users/marofke/Desktop/other.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path 'Users/marofke/Desktop/asset_ref_dir/test.txt' resolving to '/Users/marofke/Desktop/Users/marofke/Desktop/asset_ref_dir/test.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path 'Users/marofke/Desktop/asset_ref_dir/test copy.txt' resolving to '/Users/marofke/Desktop/Users/marofke/Desktop/asset_ref_dir/test copy.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path 'Users/marofke/Desktop/asset_ref_dir/does not exist.txt' resolving to '/Users/marofke/Desktop/Users/marofke/Desktop/asset_ref_dir/does not exist.txt' does not exist. Adding to referenced paths.
Job submission contains 18 input files totaling 20.8 KB.  All input files will be uploaded to S3 if they are not already present in the job attachments bucket.

Do you wish to proceed? [Y/n]: 

4. (CLI) Unsuccessful Submission when using --require-paths-exist flag

marofke@ Desktop % deadline bundle submit test_bundle_1 --yes --require-paths-exist
Submitting to Queue: CMF
Job submission contains misconfigured input directories and cannot be submitted. All input directories must exist.
Non-existent directories:
	./other_input_dir
	/Users/marofke/Desktop/asset_ref_dir/does_not_exist
Job submission canceled.

5. (GUI and CLI) Successfully Submitted my second job bundle that only had output directories

Test 2: Submitting With No Configured Storage Profile Location

1. (GUI) Successful Submission With Confirmation Popup

• confirmed setting the auto-accept config setting still showed this prompt

2. (CLI) Successful Submission With Confirmation Popup

• confirmed setting the auto-accept config setting still showed this prompt

marofke@ Desktop % deadline bundle submit test_bundle_1
Submitting to Queue: CMF
INFO:deadline.client.api._submit_job_bundle:Input path './other_input_dir' does not exist. Adding to referenced paths.
INFO:deadline.client.api._submit_job_bundle:Input path '/Users/marofke/Desktop/asset_ref_dir/does_not_exist' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path 'Users/marofke/Desktop/asset_ref_dir/test.txt' resolving to '/Users/marofke/Desktop/Users/marofke/Desktop/asset_ref_dir/test.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path './other.txt' resolving to '/Users/marofke/Desktop/other.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path 'Users/marofke/Desktop/asset_ref_dir/does not exist.txt' resolving to '/Users/marofke/Desktop/Users/marofke/Desktop/asset_ref_dir/does not exist.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path 'Users/marofke/Desktop/asset_ref_dir/test copy.txt' resolving to '/Users/marofke/Desktop/Users/marofke/Desktop/asset_ref_dir/test copy.txt' does not exist. Adding to referenced paths.
Job submission contains 18 input files totaling 20.8 KB.  All input files will be uploaded to S3 if they are not already present in the job attachments bucket.

Files were specified outside of the configured storage profile location(s).  Please confirm that you intend to submit a job that uses files from the following directories:

Under the directory '/Users/marofke/Desktop':
	18 input files
	2 output directories
	6 referenced files and/or directories

To permanently remove this warning you must only use files located within a storage profile location.

Do you wish to proceed? [y/N]: 

3. (GUI and CLI) Submitted my second job bundle, confirming it also prompted when output paths fell outside of a storage profile filesystem location

Test 3: Submitting Job with Directory Marked as File

1. (GUI) Confirmed it failed to submit

2. (CLI) Confirmed it failed to submit

marofke@ Desktop % deadline bundle submit test_bundle_1    
Submitting to Queue: CMF
INFO:deadline.client.api._submit_job_bundle:Input path './other_input_dir' does not exist. Adding to referenced paths.
INFO:deadline.client.api._submit_job_bundle:Input path '/Users/marofke/Desktop/asset_ref_dir/does_not_exist' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path './other.txt' resolving to '/Users/marofke/Desktop/other.txt' does not exist. Adding to referenced paths.
INFO:deadline.job_attachments.upload:Input path '/Users/marofke/Desktop/asset_ref_dir/does not exist.txt' resolving to '/Users/marofke/Desktop/asset_ref_dir/does not exist.txt' does not exist. Adding to referenced paths.
Job submission contains missing input files or directories specified as files. All inputs must exist and be classified properly.
Directories classified as files:
	/Users/marofke/Desktop/asset_ref_dir/nested
Job submission canceled.

UI Changes

• Here's the new UI setting that allows users to enforce all paths exist:

  Show / Hide

  

Was this change documented?

No

Is this a breaking change?

Yes, added new CLI flag and reworked the public interface of the Job Attachments S3AssetUploader

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[chocecil]

Why isn't this path checking enabled by default? It appears to be quite useful, and it would be beneficial if we could always validate it.

[marofke]

Why isn't this path checking enabled by default? It appears to be quite useful, and it would be beneficial if we could always validate it.

Mark and I talked about this a bit before I made the change, and while we agreed enabling by default would be ideal, doing so would be a break in existing behaviour (I.e. isn't backwards compatible), as we currently only log if files are missing.

[chocecil]

Why isn't this path checking enabled by default? It appears to be quite useful, and it would be beneficial if we could always validate it.

Mark and I talked about this a bit before I made the change, and while we agreed enabling by default would be ideal, doing so would be a break in existing behaviour (I.e. isn't backwards compatible), as we currently only log if files are missing.

Make sense.
I was thinking just printing warning messages but proceeding with submission isn't a non backwards change even when the flag is not given. The example in 3. (CLI) Successful Submission With Confirmation Prompt in the description seems to do that exactly.