test: Add test to verify WA job attachment settings #373
Conversation
| job_bundle_path: str = os.path.join( | ||
| os.path.dirname(__file__), | ||
| "..", | ||
| "..", | ||
| "..", | ||
| "scripts", | ||
| "submit_jobs", | ||
| "job_attachment_job", | ||
| ) |
There was a problem hiding this comment.
I'd suggest putting the test job beside this file, rather than back up in the scripts directory. Reason being that the scripts directory is intended to be a fairly random collection of useful scripts and files for manual testing. Putting a part of the e2e tests in there risks someone changing the bundle in some way that's unexpected by the test. Yes, the test should catch that, but it's a better idea to have the testing files all within the testing directories, imo.
There was a problem hiding this comment.
Done, I've put the job bundle under the tests path
|
|
||
| assert output_file_content == (input_file_content + test_run_uuid) | ||
|
|
||
| if os.path.exists( |
There was a problem hiding this comment.
Rather than this, I'd suggest downloading the output files into a temporary directory. Reason being that a failed test will leave files behind right now -- e.g. if the output_file.txt doesn't get generated, or downloaded, for some reason then the open call above will raise an exception and abort the test.
If you use a tempfile.TemporaryDirectory as a context manager, instead, then the tempdir will get deleted regardless of whether the test exists successfully or not.
Alternatively to that, there's also the tmp_path fixture built-in to pytest for getting a temporary directory for the duration of a test: https://docs.pytest.org/en/stable/how-to/tmp_path.html#the-tmp-path-fixture
There was a problem hiding this comment.
I've changed it to use a temp directory in the revision. Thanks for pointing this out
Signed-off-by: Yutong Li <52769999+YutongLi291@users.noreply.github.com>
|
What was the problem/requirement? (What/Why)
We want to verify that worker agent is able to follow queues job attachment settings and correctly identify input/output directories to take actions on files.
We should add testing regarding this functionality.
What was the solution? (How)
Add a test that verifies that WA is able to execute a job that respects the job attachment settings, and verifies that the job output is as we expected.
What is the impact of this change?
Better verification on the functionality to use job attachments with jobs, as well as with Worker Agent.
How was this change tested?
hatch run linux-e2e-tests
Was this change documented?
no
Is this a breaking change?
no
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.