Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Submitter Callback functions #132

Open
wants to merge 6 commits into
base: mainline
Choose a base branch
from

Conversation

Ahuge
Copy link

@Ahuge Ahuge commented Apr 19, 2024

What was the problem/requirement? (What/Why)

When working in a studio, we often want to execute local validation of scene files before we allow a submission to the render farm. In other scenarios we may need to modify or transform the scene to prepare if for the render farm.

These two hooks allow for studio specified modification hooks to be optionally run before and after a submission completes.

What was the solution? (How)

This solution implements an on_pre_submit_callback and a on_post_submit_callback that take the same parameters as the on_create_job_bundle_callback.
The parameters match on_create_job_bundle_callback solely to provide flexibility in case the hook needs access to any of the settings of the job.

What is the impact of this change?

This change relies on my PR in deadline-cloud which implements an on_pre_submit_callback and a on_post_submit_callback parameter and functionality.

This PR just implements loading of the callback from an environment variable

How was this change tested?

Did you run the "Job Bundle Output Tests"? If not, why not? If so, paste the test results here.

Have not yet, will do

Required: paste the contents of job_bundle_output_tests/test-job-bundle-results.txt here

Was this change documented?

Not currently documented

Is this a breaking change?

No it is a feature addition


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Ahuge Ahuge requested a review from a team as a code owner April 19, 2024 00:37
@Ahuge Ahuge force-pushed the ah/feature/submit_callback_functions branch from 23c63a9 to 2768e7c Compare April 19, 2024 00:40
@Ahuge Ahuge changed the title Submitter Callback functions Add Submitter Callback functions Apr 19, 2024
@Ahuge Ahuge changed the title Add Submitter Callback functions feat: Add Submitter Callback functions Apr 19, 2024
@Ahuge Ahuge force-pushed the ah/feature/submit_callback_functions branch from 2768e7c to 111433b Compare April 19, 2024 15:20
@Ahuge
Copy link
Author

Ahuge commented Apr 19, 2024

So to explain this PR a bit more.

The base implementation (in 985ca84) adds a check for the presence of an environment variable for pre and post submit hooks.
If found, it dynamically loads that file off of disk and tries to load a specific function within it.
It uses a new function within deadline-cloud that I also added to do this functionality.
It then passes those functions down to the deadline-cloud SubmitJobToDeadlineDialog dialog.


The second commit, 72035a9, just adds some exception handling around this dynamic loading. I figure that throwing a DeadlineOperationError could then be caught somewhere for the user.


The third commit, 111433b, is where I had an idea to implement some function validation at this point to ensure the function that we are loading matches the signature that we expect.
This relies on an idea I had in the deadline-cloud PR where it would implement a function that checks the function signature of a "callback" and ensures it matches what we would expect.

This does some python inspect magic to check the function signature and may be perceived as superfluous.
I feel like this additional signature validation step is functionality that I will leave up to the maintainers to decide if they would like to include.


And lastly the I change the environment variables from DEADLINE_PRE_SUBMIT_CALLBACK to DEADLINE_NUKE_PRE_SUBMIT_CALLBACK because I expect we will want "per-DCC" callback support

@Ahuge Ahuge force-pushed the ah/feature/submit_callback_functions branch from a0fe521 to 8eaf450 Compare April 26, 2024 22:23
Ahuge added 5 commits May 9, 2024 12:53
This implementation looks for `DEADLINE_PRE_SUBMIT_CALLBACK` and `DEADLINE_POST_SUBMIT_CALLBACK` which should point to a python file that contains a function called `on_pre_submit_callback` and `on_post_submit_callback` respectively.

 The new `nuke_util.callback` file contains `import_module_function` which will load the file from disk.
 This has no "safeties" on it to catch exceptions during import perhaps we should add some exception handling around this function specifically.

Signed-off-by: Alex Hughes <ahughesalex@gmail.com>
Signed-off-by: Alex Hughes <ahughesalex@gmail.com>
…ction

This will call the `validate_function_signature` function within the new deadline-cloud callback loader to validate the parameters of the loaded callback.py
This is optional as I am unsure how much we care about this. It would fail anyways but this way at least we could raise some error notifying people that the signature looks incorrect.

Signed-off-by: Alex Hughes <ahughesalex@gmail.com>
This modifies the environment variable name to include the DCC that it applies to.
You may want a callback per-dcc so this adds flexibility

Signed-off-by: Alex Hughes <ahughesalex@gmail.com>
Signed-off-by: Alex Hughes <ahughesalex@gmail.com>
@Ahuge Ahuge force-pushed the ah/feature/submit_callback_functions branch from 8eaf450 to 3fbd40a Compare May 9, 2024 19:54
…e as well.

Signed-off-by: Alex Hughes <ahughesalex@gmail.com>
Copy link

sonarcloud bot commented May 17, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant