Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: job fails if Write node has no folders in filename #14

Merged
merged 19 commits into from
Sep 14, 2023

Conversation

amznestebanpapp
Copy link
Contributor

What was the problem/requirement? (What/Why)

When submitting a job in Nuke that has a Write node with a filename that does not contain a folder, the job fails.
Example nk:

Root {
 inputs 0
 name colorbars.nk
 frame -3
 format "2048 1556 0 0 2048 1556 1 2K_Super_35(full-ap)"
 proxy_type scale
 proxy_format "1024 778 0 0 1024 778 1 1K_Super_35(full-ap)"
 colorManagement Nuke
 workingSpaceLUT linear
 monitorLut sRGB
 monitorOutLUT rec709
 int8Lut sRGB
 int16Lut sRGB
 logLut Cineon
 floatLut linear
}
ColorBars {
 inputs 0
 barintensity {{curve x1 0 x10 1}}
 name ColorBars1
 xpos -444
 ypos -375
}
set N6b5d800 [stack 0]
Viewer {
 frame -3
 frame_range 1-100
 monitorOutOutputTransform rec709
 name Viewer1
 xpos -444
 ypos -218
}
push $N6b5d800
Write {
 file output_####.exr
 file_type exr
 first_part rgba
 name Write1
 xpos -296
 ypos -259
}

Error:

2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR: Traceback (most recent call last):
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:   File "/tmp/ojio/session-c3d2e8510f7a4d1c879653acad8b2319rshv_oqi/session-env/packages/nuke/13.2/platform-linux/plugins/nuke_internal/callbacks.py", line 173, in beforeRender
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:     _doCallbacks(beforeRenders)
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:   File "/tmp/ojio/session-c3d2e8510f7a4d1c879653acad8b2319rshv_oqi/session-env/packages/nuke/13.2/platform-linux/plugins/nuke_internal/callbacks.py", line 46, in _doCallbacks
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:     f[0](*f[1],**f[2])
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:   File "/tmp/ojio/session-c3d2e8510f7a4d1c879653acad8b2319rshv_oqi/session-env/packages/deadline_cloud_for_nuke/0.9.0/_v/a/python/deadline/nuke_adaptor/NukeClient/nuke_client.py", line 48, in ensure_output_dir
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:     os.makedirs(output_dir)
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:   File "/tmp/ojio/session-c3d2e8510f7a4d1c879653acad8b2319rshv_oqi/session-env/packages/nuke/13.2/platform-linux/lib/python3.7/os.py", line 223, in makedirs
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR:     mkdir(name, mode)
2023/09/06 17:18:10-07:00 ADAPTOR_OUTPUT: STDERR: FileNotFoundError: [Errno 2] No such file or directory: ''

What was the solution? (How)

Check that there is a folder in the filename before trying to create one.

What is the impact of this change?

Despite uncommon in practice, a Write node does not necessarily has to contain a folder. A user could indicate an output filename without a folder.

How was this change tested?

Above nk script was used to submit jobs. A job with a filename that does not contain a folder was submitted successful with these changes.
Other changes were done to be able to upload wheels (also tested).

Did you run the "Job Bundle Output Tests"? If not, why not? If so, paste the test results here.

The difference is because I am using a newer version of Nuke. No other differences found:

Timestamp: 2023-09-07T19:06:49.873239-07:00
Running job bundle output test: D:\ws\deadline-cloud-for-nuke\job_bundle_output_tests\multi-load-save

multi-load-save
Test failed, found differences
--- expected/parameter_values.yaml
+++ test/parameter_values.yaml
@@ -14,6 +14,6 @@
 - name: ProxyMode
   value: 'false'
 - name: NukeVersion
-  value: 13.2v4
+  value: 14.0v5
 - name: RezPackages
   value: nuke-13 deadline_cloud_for_nuke

Timestamp: 2023-09-07T19:18:28.632054-07:00
Running job bundle output test: D:\ws\deadline-cloud-for-nuke\job_bundle_output_tests\noise-saver

noise-saver
Test failed, found differences
--- expected/parameter_values.yaml
+++ test/parameter_values.yaml
@@ -14,6 +14,6 @@
 - name: ProxyMode
   value: 'false'
 - name: NukeVersion
-  value: 13.2v4
+  value: 14.0v5
 - name: RezPackages
   value: nuke-13 deadline_cloud_for_nuke

Failed 2 tests, succeeded 0.
Timestamp: 2023-09-07T19:19:00.837552-07:00

Was this change documented?

No

Is this a breaking change?

Yes, this is updating the dependencies to deadline-cloud and openjobio, its also changing the module name to openjd since that is what is being generated.

@amznestebanpapp amznestebanpapp requested a review from a team as a code owner September 8, 2023 02:21
pyproject.toml Outdated
"deadline == 0.20.*",
"openjobio == 0.8.*",
"deadline == 0.21.*",
"openjd == 0.10.*",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to wait for stable closed-beta installers before we merge this dep update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will revert these changes for the time being

DEVELOPMENT.md Outdated Show resolved Hide resolved
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
scripts/build_wheels.sh Show resolved Hide resolved
@@ -228,6 +228,7 @@ def _run_job_bundle_output_test(test_dir: str, dcc_scene_file: str, report_fh, m
)
contents = contents.replace(tempdir, "/normalized/job/bundle/dir")
contents = contents.replace(tempdir.replace("\\", "/"), "/normalized/job/bundle/dir")
contents = contents.replace(os.getcwd(), "/normalized/cwd")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect we should explicitly set the CWD to a known temporary directory different from the job bundle dir, to ensure this has predictable behavior.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I am following the comment.

the CWD in my case was no the job bundle dir, but the path where I launch Nuke from (which happened to be the root of this repo).
This currently works because os.getcwd does not change from the moment job attachments collects the paths to this point.

Can you clarify what is the unpredictable behavior you are concerned about?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CWD depends on how the user launched Nuke. It might contain an arbitrary set of files in it, that could be read or ignored by Nuke or scripts embedded in a Nuke scene. Creating a fresh temporary directory, and setting the CWD to it before running the tests eliminates that potential variability.

Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Copy link
Contributor

@rmv rmv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

return nuke.root().knob("project_directory").getEvaluatedValue()
project_path = nuke.root().knob("project_directory").getEvaluatedValue()
if not project_path:
project_path = os.getcwd()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to set this to the directory of the nuke.scriptName() instead of the directory from which Nuke was launched?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nuke takes cwd as that path. Meaning, if we didnt define project_directory and have a relative file path in a Write node. The file will go to cwd.
So here we have to take the same approach, otherwise you could have project_directory undefined and a relative path in a Read node and assume that you are relative to nuke.scriptName, but Nuke is realtive to cwd.
I have seen some cases where TDs set this up relative to the cwd and have artists launch the tool from a specific path (e.g. from within the "shot" path).

@amznestebanpapp amznestebanpapp changed the title Job fails if Write node has no folders in filename fix: job fails if Write node has no folders in filename Sep 12, 2023
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
Signed-off-by: Esteban Papp <81431996+amznestebanpapp@users.noreply.github.com>
@amznestebanpapp amznestebanpapp merged commit 294f150 into mainline Sep 14, 2023
5 checks passed
@amznestebanpapp amznestebanpapp deleted the pappeste/no_output_folder branch September 14, 2023 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants