refactor!: Modify the Maya integrated submitter to use job params

BREAKING CHANGES:
- This updates the implementation to use the updated adaptor support
interface
- Rearrange the high level directory structure so that there is a clean
  submitter_plugin directory that holds the plugin structure Maya needs,
  and a clean usage of the deadline namespace package for the bulk of
  the adaptor and submitter code.
- Change the Camera selection options to "All Cameras" and the specific
  cameras that are applicable to the current layer selection.

Other Notes:
- Enabled the camera options, whether to render a layer's cameras
in one task or multiple tasks
- The Job Parameters now include MayaSceneFile, Frames,
RezPackages, and more.
- Remove the pattern "import <module> as _<module>"
- Place the Maya integrated submitter into the deadline namespace
  package
- Add a developer option that can submit wheels along with the job to
  override the adaptor, and describe how to enable it in DEVELOPMENT.md.
- Change the default max failed tasks limit from 100 to 20, to reflect
  discussion on the topic.
- Remove tests that are mocking Maya in favour of adding an in-Maya job
  bundle test mechanism.
- Update the icon in Maya, it was failing to load.
- Reworked shelf.py to use maya.mel and maya.cmds instead of pymel.
  Also removed the custom shelf.json and code that parsed and
  interpreted it. The code is much simpler to create the shelf and
  buttons directly.
- Removed all usage of pymel from the submitter and adaptor.
- Removed all unit tests that were based on mocking pymel calls.
- Update the install_builder xml as best I can without running it.

Signed-off-by: Mark Wiebe <markw@amazon.com>

.github/PULL_REQUEST_TEMPLATE.md

@@ -6,6 +6,12 @@
 
 ### How was this change tested?
 
+### Did you run the "Job Bundle Output Tests"? If not, why not? If so, paste the test results here.
+
+```
+Required: paste the contents of job_bundle_output_tests/test-job-bundle-results.txt here
+```
+
 ### Was this change documented?
 
-### Is this a breaking change?
+### Is this a breaking change?

.gitignore

@@ -8,3 +8,6 @@ __pycache__/
 .vscode
 *_version.py
 dependency_bundle
+/wheels/
+**/test_job_bundle/
+**/test-job-bundle-results.txt

DEVELOPMENT.md

@@ -16,20 +16,54 @@ All of the testing and development dependencies are in the `setup.cfg` and are m
 
 `./build.sh` will build and run `pytest`.
 
-# Windows Development Workflow
+# Submitter Development Workflow
 
 WARNING: This workflow installs additional Python packages into your Maya's python distribution.
 
 1. Create a development location within which to do your git checkouts. For example `~/deadline-clients`.
    Clone packages from this directory with commands like
-   `git clone git@github.com:casillas2/deadline-maya.git`. You'll also want the `deadline` repo.
+   `git clone git@github.com:casillas2/deadline-cloud-for-maya.git`. You'll also want the `deadline-cloud` and `openjobio` repos.
 2. Switch to your Maya directory, like `cd "C:\Program Files\Autodesk\Maya2023"`.
-3. Run `.\mayapy -m pip install -e C:\Users\<username>\deadline-clients\deadline` to install the Amazon Deadline Cloud Client
+3. Run `.\mayapy -m pip install -e C:\Users\<username>\deadline-clients\deadline-cloud` to install the Amazon Deadline Cloud Client
    Library in edit mode.
-4. Run `.\mayapy -m pip install -e C:\Users\<username>\deadline-clients\deadline-nuke` to install the Nuke Submtiter
+4. Run `.\mayapy -m pip install -e C:\Users\<username>\deadline-clients\openjobio` to install the OpenJobIO
+   Library in edit mode.
+5. Run `.\mayapy -m pip install -e C:\Users\<username>\deadline-clients\deadline-cloud-for-maya` to install the Maya Submtiter
    in edit mode.
-6. Edit (create if missing) your `~/Documents/maya/2023/Maya.env` file, and add the line
-   `MAYA_MODULE_PATH=C:\Users\<username>\deadline-clients\deadline-maya\src\deadline_submitter_for_maya`
+6. Edit (create if missing) your `~/Documents/maya/2023/Maya.env` file, and add the following lines to it:
+   ```
+   DEADLINE_ENABLE_DEVELOPER_OPTIONS=true
+   MAYA_MODULE_PATH=C:\Users\<username>\deadline-clients\deadline-cloud-for-maya\maya_submitter_plugin
+   ```
+   The developer options add a shelf TEST button you can use to run the tests from the `job_bundle_output_tests` directory.
 7. Run Maya. Go to Windows > Settings/Preferences > Plug-In Manager and you will find that
-   `DeadlineSubmitter.py` is available as a plugin. Check the checkbox to have Maya load
+   `DeadlineCloudForMaya.py` is available as a plugin. Check the checkbox to have Maya load
    the plugin and create the Deadline tray for you. Click the icon on the tray to open the submitter.
+
+# Application Interface Adaptor Development Workflow
+
+You can work on the adaptor alongside your submitter development workflow using a Deadline Cloud
+farm that uses a service-managed fleet. You'll need to perform the following steps to subsititute
+your build of the adaptor for the one in the service.
+
+1. Use the development location from the Submitter Development Workflow.
+   Make sure you're running Maya with `set DEADLINE_ENABLE_DEVELOPER_OPTIONS=true` enabled.
+2. Build wheels for `openjobio`, `deadline` and `deadline-cloud-for-maya`.
+   ```
+   # If you don't have the build package installed already
+   $ pip install build
+   ...
+   $  mkdir wheels; \
+      rm wheels/*; \
+      for dir in ../openjobio ../deadline-cloud ../deadline-cloud-for-maya; \
+        do python -m build --wheel --outdir ./wheels --skip-dependency-check $dir; \
+      done
+   ...
+   $ ls ./wheels
+   deadline_cloud_for_maya-<version>-py3-none-any.whl
+   deadline-<version>-py3-none-any.whl
+   openjobio-<version>-py3-none-any.whl
+   ```
+3. Open the Maya integrated submitter, and in the Job-Specific Settings tab, enable the option 'Include Adaptor Wheels'. This
+   option is only visible when the environment variable `DEADLINE_ENABLE_DEVELOPER_OPTIONS` is set to `true`.
+   Then submit your test job.

README.md

@@ -5,7 +5,7 @@ This package has two active branches:
 - `mainline` -- For active development. This branch is not intended to be consumed by other packages. Any commit to this branch may break APIs, dependencies, and so on, and thus break any consumer without notice.
 - `release` -- The official release of the package intended for consumers. Any breaking releases will be accompanied with an increase to this package's interface version.
 
-The deadline_adaptor_for_maya package is an adaptor that renders maya scenes through MayaPy and PyMel. It uses the openjobio_adaptor_runtime and supports job stickiness.
+The deadline.maya_adaptor package is an adaptor that renders maya scenes through MayaPy. It uses the openjobio adaptor_runtime and supports job stickiness.
 
 ## Development
 
@@ -43,4 +43,3 @@ See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more inform
 ## License
 
 This project is licensed under the Apache-2.0 License.
-

hatch.toml

@@ -5,8 +5,8 @@ pre-install-commands = [
 
 [envs.default.scripts]
 sync = "pip install -r requirements-testing.txt"
-test = "pytest --cov-config pyproject.toml {args:test/*/unit}"
-typing = "mypy {args:src test}"
+test = "pytest --cov-config pyproject.toml {args:test}"
+typing = "mypy {args:src test maya_submitter_plugin/plug-ins}"
 style = [
   "ruff {args:.}",
   "black --check --diff {args:.}",
@@ -38,4 +38,4 @@ SKIP_BOOTSTRAP_TEST_RESOURCES="True"
 test = "pytest --no-cov {args:test/*/integ}"
 
 [envs.e2e.scripts]
-test = "pytest --no-cov {args:test/*/e2e}"
+test = "pytest --no-cov {args:test/*/e2e}"

hatch_custom_hook.py

@@ -0,0 +1,47 @@
+# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+import os
+import shutil
+
+from hatchling.builders.hooks.plugin.interface import BuildHookInterface
+from typing import Any
+
+
+class HatchCustomBuildHook(BuildHookInterface):
+    """
+    This class implements Hatch's [custom build hook] (https://hatch.pypa.io/1.6/plugins/build-hook/custom/)
+    for a copy_version_py operation that copies the _version.py file generated by the hatch-vcs build hook into
+    specified destination directories. See the `[[tool.hatch.build.hooks.custom]]` section in `pyproject.toml`.
+    """
+
+    def _validate_config(self):
+        if sorted(self.config) != ["copy_version_py", "path"] or list(
+            self.config["copy_version_py"]
+        ) != ["destinations"]:
+            raise RuntimeError(
+                "Configuration of the custom build hook must be like { 'copy_version_py': {'destinations': ['path1', ...]}}."
+                + f" Received:\n{self.config}"
+            )
+
+    def initialize(self, version: str, build_data: dict[str, Any]) -> None:
+        self._validate_config()
+
+        for destination in self.config["copy_version_py"]["destinations"]:
+            print(f"Copying _version.py to {destination}")
+            shutil.copy(
+                os.path.join(self.root, "_version.py"),
+                os.path.join(self.root, destination),
+            )
+
+    def clean(self, versions: list[str]) -> None:
+        self._validate_config()
+
+        cleaned_count = 0
+        for destination in self.config["copy_version_py"]["destinations"]:
+            print(f"Cleaning _version.py from {destination}")
+            clean_path = os.path.join(self.root, destination, "_version.py")
+            try:
+                os.remove(clean_path)
+                cleaned_count += 1
+            except FileNotFoundError:
+                pass
+        print(f"Cleaned {cleaned_count} items")

install_builder/deadline-cloud-for-maya.xml

@@ -1,20 +1,30 @@
 <component>
-    <name>maya_integrated_submitter</name>
-    <description>Integrated submitter for Maya</description>
-	<detailedDescription>Maya plugin for submitting work to Deadline</detailedDescription>
+    <name>deadline_cloud_for_maya</name>
+    <description>Deadline Cloud for Maya</description>
+	<detailedDescription>Maya plugin for submitting jobs to Amazon Deadline Cloud.</detailedDescription>
     <canBeEdited>1</canBeEdited>
     <selected>0</selected>
     <show>1</show>
     <folderList>
         <folder>
-            <description>Program Files</description>
+            <description>Maya Plug-in Module</description>
             <destination>${maya_installdir}</destination>
             <name>maya</name>
             <platforms>all</platforms>
             <distributionFileList>
                 <distributionDirectory allowWildcards="1">
-                    <origin>components/deadline-cloud-for-maya/src/deadline_cloud_for_maya_submitter/*</origin>
-                    <excludeFiles>*/install_builder;components/deadline-cloud-for-maya/src/deadline_cloud_for_maya_submitter/plugins/__init__.py</excludeFiles>
+                    <origin>components/deadline-cloud-for-maya/maya_submitter_plugin</origin>
+                </distributionDirectory>
+            </distributionFileList>
+        </folder>
+        <folder>
+            <description>Maya Submitter Files</description>
+            <destination>${maya_installdir}/scripts/deadline/maya_submitter</destination>
+            <name>maya</name>
+            <platforms>all</platforms>
+            <distributionFileList>
+                <distributionDirectory allowWildcards="1">
+                    <origin>components/deadline-cloud-for-maya/src/deadline/maya_submitter</origin>
                 </distributionDirectory>
             </distributionFileList>
         </folder>
@@ -62,14 +72,14 @@
 	</readyToInstallActionList>
 	<parameterList>
 		<stringParameter name="maya_integrated_submitter_summary" ask="0" cliOptionShow="0">
-			<value>Maya Submitter
-- Install the integrated Maya submitter files to the installation directory
-- Create/update the MAYA_MODULE_PATH environment variable to point to the integrated Maya submitter files</value>
+			<value>Deadline Cloud for Maya
+- Install the integrated Maya submitter files to the installation directory.
+- Register the plug-in with Maya by creating or updating the MAYA_MODULE_PATH environment variable.</value>
 		</stringParameter>
 	</parameterList>
     <postInstallationActionList>
 		<fnAddPathEnvironmentVariable>
-			<progressText>Setting MAYA_MODULE_PATH</progressText>
+			<progressText>Registering plug-in with Maya</progressText>
 			<name>MAYA_MODULE_PATH</name>
 			<value>${maya_installdir}</value>
 			<scope>${installscope}</scope>

job_bundle_output_tests/cube/expected_job_bundle/asset_references.yaml

@@ -0,0 +1,10 @@
+assetReferences:
+  inputs:
+    directories: []
+    filenames:
+    - /normalized/job/bundle/dir/cube.ma
+    - /normalized/job/bundle/dir/deadline_logo_48x48.png
+    - /normalized/job/bundle/dir/scene_file_to_reference.mb
+  outputs:
+    directories:
+    - /normalized/job/bundle/dir/images