diff --git a/.github/workflows/deploy_docker_images.yml b/.github/workflows/deploy_docker_images.yml new file mode 100644 index 00000000..65f34a43 --- /dev/null +++ b/.github/workflows/deploy_docker_images.yml @@ -0,0 +1,28 @@ +name: 🐳 Docker Image Deployment + +on: + workflow_dispatch: + release: + types: [published] + +jobs: + build: + name: Push docker images to docker hub + runs-on: ubuntu-latest + steps: + - name: Check out the repo + uses: actions/checkout@v4 + + - name: Login to Docker Hub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_PASSWORD }} + + - name: Build service images + id: build-docker-images + run: make build-all SERVICE=leecher -C ./services + + - name: Push service images + id: push-docker-images + run: make dist-all SERVICE=leecher -C ./services diff --git a/.github/workflows/pr_linting.yml b/.github/workflows/pr_linting.yml new file mode 100644 index 00000000..3d2431b6 --- /dev/null +++ b/.github/workflows/pr_linting.yml @@ -0,0 +1,24 @@ +name: 📇 Code Linting + +on: + push: + branches: [ develop ] + pull_request: + branches: [ develop ] + + workflow_dispatch: + +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number}} + cancel-in-progress: true + +permissions: + contents: read + pull-requests: write + +jobs: + linting: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: chartboost/ruff-action@v1 diff --git a/.github/workflows/release_trigger.yml b/.github/workflows/release_trigger.yml new file mode 100644 index 00000000..4293e4a8 --- /dev/null +++ b/.github/workflows/release_trigger.yml @@ -0,0 +1,25 @@ +name: 🚀 Release Trigger + +on: + workflow_dispatch: + inputs: + draft: + type: boolean + description: "Create Release Draft" + required: false + default: false + release_overwrite: + type: string + description: "Set Version Release Tag" + required: false + +jobs: + call-release-trigger: + uses: ynput/ops-repo-automation/.github/workflows/release_trigger.yml@main + with: + draft: ${{ inputs.draft }} + release_overwrite: ${{ inputs.release_overwrite }} + secrets: + token: ${{ secrets.YNPUT_BOT_TOKEN }} + email: ${{ secrets.CI_EMAIL }} + user: ${{ secrets.CI_USER }} diff --git a/.github/workflows/upload_to_ynput_cloud.yml b/.github/workflows/upload_to_ynput_cloud.yml new file mode 100644 index 00000000..7745a8e0 --- /dev/null +++ b/.github/workflows/upload_to_ynput_cloud.yml @@ -0,0 +1,16 @@ +name: 📤 Upload to Ynput Cloud + +on: + workflow_dispatch: + release: + types: [published] + +jobs: + call-upload-to-ynput-cloud: + uses: ynput/ops-repo-automation/.github/workflows/upload_to_ynput_cloud.yml@main + secrets: + CI_EMAIL: ${{ secrets.CI_EMAIL }} + CI_USER: ${{ secrets.CI_USER }} + YNPUT_BOT_TOKEN: ${{ secrets.YNPUT_BOT_TOKEN }} + YNPUT_CLOUD_URL: ${{ secrets.YNPUT_CLOUD_URL }} + YNPUT_CLOUD_TOKEN: ${{ secrets.YNPUT_CLOUD_TOKEN }} diff --git a/README.md b/README.md index 81392e23..1bb9c78f 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ -# Shotgrid integration for Ayon +# Shotgrid integration for AYON -This project provides three elements for the Ayon pipeline: - * server/ - The Ayon Backend Addon. +This project provides three elements for the AYON pipeline: + * server/ - The AYON Backend Addon. * frontend/ - The AYON server Shotgrid settings tab. * client/ - The AYON desktop integration. * services/ - Standalone dockerized daemons that act based on events (aka `leecher` and `processors`). @@ -22,7 +22,7 @@ We can now go into the `Settings > Studio settings > Shotgrid` page in AYON and ## Desktop application -When launching Ayon for the first time you'll be asked to provide a login (only the username) for Shotgrid, this is the user that will be used for publishing. +When launching AYON for the first time you'll be asked to provide a login (only the username) for Shotgrid, this is the user that will be used for publishing. After providing a login people can publish normally, the integartion will ensure that the user can connect to Shotgrid, that has the correct permissions and will create the Version and PublishedFile in Shotgrid if the publish is succesful. ## Services @@ -35,7 +35,7 @@ The three provided services are: * `leecher` - Periodically queries the `EventLogEntry` table on Shotgrid and ingests any event that interests us dispatching it as a `shotgrid.event`, this will only query projects that have the "Ayon Auto Sync" field enabled. * `transmitter` - Periodically check for new events in AYON of topic `entity.*`, and push any changes to Shotgrid, only affects to projects that have the "Ayon Auto Sync" field enabled. -The most straighforward way to get this up and running is by using ASH (Ayon Service Host), after loading the Addon on the server, you should be able to spawn services in the "Services" page. +The most straighforward way to get this up and running is by using ASH (AYON Service Host), after loading the Addon on the server, you should be able to spawn services in the "Services" page. ### Development There's a single `Makefile` at the root of the `services` folder, which is used to `build` the docker images and to run the services locally with the `dev` target, this is UNIX only for the time being, running `make` without argument will print information as to how to run use it. @@ -46,7 +46,7 @@ To build the docker images you can run `make SERVICE= build`, so f #### Running the Service locally In order to run the service locally we need to specify certain environment variables, to do so, copy the `sample_env` file, rename to `.env` and fill the fields acordingly: ``` -AYON_API_KEY= # You can create a `service` user in Ayon, and then get the Key from there. +AYON_API_KEY= # You can create a `service` user in AYON, and then get the Key from there. AYON_SERVER_URL= PYTHONDONTWRITEBYTECODE=1 ``` @@ -60,7 +60,7 @@ You should now see something similar to: ```sh INFO Initializing the Shotgrid Processor. DEBUG Found these handlers: {'create-project': [], 'sync-from-shotgrid': [], 'shotgrid-event': []} -INFO Start enrolling for Ayon `shotgrid.event` Events... +INFO Start enrolling for AYON `shotgrid.event` Events... INFO Querying for new `shotgrid.event` events... INFO No event of origin `shotgrid.event` is pending. ``` diff --git a/client/ayon_shotgrid/lib/credentials.py b/client/ayon_shotgrid/lib/credentials.py index ea020c45..92b44028 100644 --- a/client/ayon_shotgrid/lib/credentials.py +++ b/client/ayon_shotgrid/lib/credentials.py @@ -138,7 +138,7 @@ def save_local_login(username, password): reg = AYONSecureRegistry("shotgrid/user") reg.set_item("value", username) reg = AYONSecureRegistry("shotgrid/pass") - reg.set_item("value", password) + reg.set_item("value", password or "") def clear_local_login(): diff --git a/client/ayon_shotgrid/plugins/publish/integrate_shotgrid_version.py b/client/ayon_shotgrid/plugins/publish/integrate_shotgrid_version.py index 38c00291..59597781 100644 --- a/client/ayon_shotgrid/plugins/publish/integrate_shotgrid_version.py +++ b/client/ayon_shotgrid/plugins/publish/integrate_shotgrid_version.py @@ -16,7 +16,7 @@ class IntegrateShotgridVersion(pyblish.api.InstancePlugin): label = "Shotgrid Version" # Dictionary of SG fields we want to update that map to other fields in the - # Ayon entity + # AYON entity fields_to_add = { "comment": (str, "description"), "productType": (str, "sg_version_type"), @@ -176,7 +176,7 @@ def process(self, instance): "Instance doesn't have a 'versionEntity' to extract the id." ) version_id = "-" - data_to_update["sg_ayon_id"] = version_id + data_to_update["sg_ayon_id"] = str(version_id) self.log.info(f"Updating Shotgrid version with {data_to_update}") sg_session.update("Version", sg_version["id"], data_to_update) diff --git a/client/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py b/client/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py index 7c4a9093..82509821 100644 --- a/client/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py +++ b/client/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py @@ -20,7 +20,7 @@ def process(self, context): if not (user_login and sg_session and sg_project): raise PublishValidationError("Missing Shotgrid Credentials") - self.log.info("Login ShotGrid set in Ayon is {}".format(user_login)) + self.log.info("Login ShotGrid set in AYON is {}".format(user_login)) self.log.info("Current ShotGrid Project is {}".format(sg_project)) sg_user = sg_session.find_one( diff --git a/client/ayon_shotgrid/tray/sg_login_dialog.py b/client/ayon_shotgrid/tray/sg_login_dialog.py index d495ee2b..9295540a 100644 --- a/client/ayon_shotgrid/tray/sg_login_dialog.py +++ b/client/ayon_shotgrid/tray/sg_login_dialog.py @@ -21,6 +21,7 @@ def __init__(self, addon, parent=None): self.login_type = self.addon.get_client_login_type() self.setWindowTitle("Ayon - Shotgrid Login") + icon = QtGui.QIcon(resources.get_ayon_icon_filepath()) self.setWindowIcon(icon) @@ -44,7 +45,7 @@ def setup_ui(self): server_url = self.addon.get_sg_url() if not server_url: - server_url = "No Shotgrid Server set in Ayon Settings." + server_url = "No Shotgrid Server set in AYON Settings." sg_server_url_label = QtWidgets.QLabel( "Please provide the credentials to log in into the " diff --git a/client/ayon_shotgrid/tray/shotgrid_tray.py b/client/ayon_shotgrid/tray/shotgrid_tray.py index 20880d26..044ab92c 100644 --- a/client/ayon_shotgrid/tray/shotgrid_tray.py +++ b/client/ayon_shotgrid/tray/shotgrid_tray.py @@ -7,7 +7,7 @@ class ShotgridTrayWrapper: - """ Shotgrid menu entry for the Ayon tray. + """ Shotgrid menu entry for the AYON tray. Displays the Shotgrid URL specified in the Server Addon Settings and allows the person to set a username to be used with the API. @@ -21,7 +21,7 @@ def __init__(self, addon): server_url = self.addon.get_sg_url() if not server_url: - server_url = "No Shotgrid Server set in Ayon Settings." + server_url = "No Shotgrid Server set in AYON Settings." self.sg_server_label = QtWidgets.QAction("Server: {0}".format( server_url @@ -45,13 +45,13 @@ def show_sg_username_dialog(self): self.sg_username_dialog.raise_() def tray_menu(self, tray_menu): - """Add Shotgrid Submenu to Ayon tray. + """Add Shotgrid Submenu to AYON tray. A non-actionable action displays the Shotgrid URL and the other action allows the person to set and check their Shotgrid username. Args: - tray_menu (QtWidgets.QMenu): The Ayon Tray menu. + tray_menu (QtWidgets.QMenu): The AYON Tray menu. """ shotgrid_tray_menu = QtWidgets.QMenu("Shotgrid", tray_menu) shotgrid_tray_menu.addAction(self.sg_server_label) diff --git a/client/ayon_shotgrid/version.py b/client/ayon_shotgrid/version.py index 2147325c..f6868f8e 100644 --- a/client/ayon_shotgrid/version.py +++ b/client/ayon_shotgrid/version.py @@ -1,3 +1,3 @@ # -*- coding: utf-8 -*- """Package declaring shotgrid addon version.""" -__version__ = "0.4.5-dev.1" +__version__ = "0.4.6+dev" diff --git a/create_package.py b/create_package.py index e3675f79..bc3f5496 100644 --- a/create_package.py +++ b/create_package.py @@ -264,6 +264,34 @@ def create_server_package(current_dir, output_dir, addon_output_dir, log): log.info(f"Output package can be found: {output_path}") +def _propagate_version(current_dir): + # Update client version file with version from 'package.py' + client_version_file = os.path.join( + current_dir, "client", ADDON_CLIENT_DIR, "version.py" + ) + with open(client_version_file, "w") as stream: + stream.write( + CLIENT_VERSION_CONTENT.format(ADDON_NAME, ADDON_VERSION) + ) + + # Update version in services pyproject.toml files + for service in ("leecher", "processor", "transmitter"): + service_pyproject_path = os.path.join( + current_dir, "services", f"{service}/pyproject.toml" + ) + new_lines = [] + with open(service_pyproject_path, "r") as stream: + version_found = False + for line in stream.readlines(): + if not version_found and line.startswith("version"): + line = f'version = "{ADDON_VERSION}"\n' + version_found = True + new_lines.append(line) + + with open(service_pyproject_path, "w") as stream: + stream.write("".join(new_lines)) + + def main( output_dir: Optional[str] = None, skip_zip: bool = False, @@ -277,12 +305,7 @@ def main( if not output_dir: output_dir = os.path.join(current_dir, "package") - # Update client version file with version from 'package.py' - client_version_file = os.path.join( - current_dir, "client", ADDON_CLIENT_DIR, "version.py" - ) - with open(client_version_file, "w") as stream: - stream.write(CLIENT_VERSION_CONTENT.format(ADDON_NAME, ADDON_VERSION)) + _propagate_version(current_dir) addon_output_root = os.path.join(output_dir, ADDON_NAME) addon_output_dir = os.path.join(addon_output_root, ADDON_VERSION) diff --git a/package.py b/package.py index f68ee244..3103dfa4 100644 --- a/package.py +++ b/package.py @@ -1,6 +1,6 @@ name = "shotgrid" title = "Shotgrid" -version = "0.4.5-dev.1" +version = "0.4.6+dev" client_dir = "ayon_shotgrid" services = { diff --git a/ruff.toml b/ruff.toml new file mode 100644 index 00000000..d509ee4b --- /dev/null +++ b/ruff.toml @@ -0,0 +1,74 @@ +# Exclude a variety of commonly ignored directories. +exclude = [ + ".bzr", + ".direnv", + ".eggs", + ".git", + ".git-rewrite", + ".hg", + ".ipynb_checkpoints", + ".mypy_cache", + ".nox", + ".pants.d", + ".pyenv", + ".pytest_cache", + ".pytype", + ".ruff_cache", + ".svn", + ".tox", + ".venv", + ".vscode", + "__pypackages__", + "_build", + "buck-out", + "build", + "dist", + "node_modules", + "site-packages", + "venv", +] + +# Same as Black. +line-length = 79 +indent-width = 4 + +[lint] +# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default. +# Unlike Flake8, Ruff doesn't enable pycodestyle warnings (`W`) or +# McCabe complexity (`C901`) by default. +select = ["E4", "E7", "E9", "F", "W"] +ignore = [] + +# Allow fix for all enabled rules (when `--fix`) is provided. +fixable = ["ALL"] +unfixable = [] + +# Allow unused variables when underscore-prefixed. +dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$" + +[format] +# Like Black, use double quotes for strings. +quote-style = "double" + +# Like Black, indent with spaces, rather than tabs. +indent-style = "space" + +# Like Black, respect magic trailing commas. +skip-magic-trailing-comma = false + +# Like Black, automatically detect the appropriate line ending. +line-ending = "auto" + +# Enable auto-formatting of code examples in docstrings. Markdown, +# reStructuredText code/literal blocks and doctests are all supported. +# +# This is currently disabled by default, but it is planned for this +# to be opt-out in the future. +docstring-code-format = false + +# Set the line length limit used when formatting code snippets in +# docstrings. +# +# This only has an effect when the `docstring-code-format` setting is +# enabled. +docstring-code-line-length = "dynamic" diff --git a/server/__init__.py b/server/__init__.py index 49058a6a..fe3009c5 100644 --- a/server/__init__.py +++ b/server/__init__.py @@ -27,7 +27,7 @@ async def setup(self): self.request_server_restart() async def create_shotgrid_attributes(self) -> bool: - """Make sure Ayon has the `shotgridId` and `shotgridPath` attributes. + """Make sure AYON has the `shotgridId` and `shotgridPath` attributes. Returns: bool: 'True' if an attribute was created or updated. diff --git a/services/Makefile b/services/Makefile index 081d180b..2cf668e8 100644 --- a/services/Makefile +++ b/services/Makefile @@ -4,7 +4,7 @@ SERVICE = $(error Please specify the service to build with 'SERVICE', for exampl default: @echo "" - @echo "Ayon Shotgrid $(ADDON_VERSION) Service Builder" + @echo "AYON Shotgrid $(ADDON_VERSION) Service Builder" @echo "" @echo "Usage: make SERVICE=[service-name] [target]" @echo "" @@ -17,16 +17,28 @@ default: @echo "Targets:" @echo " build Build docker image." @echo " build-all Build docker image for 'leecher', 'procesor' and 'transmitter'." + @echo " dist Push docker image to DockerHub." + @echo " dist-all Push docker image for 'leecher', 'procesor' and 'transmitter'." @echo " clean Remove local images." @echo " clean-build Remove local images and build without docker cache." @echo " dev Run a service locally" build: - docker build --network=host -t $(IMAGE) -f $(SERVICE)/Dockerfile . + docker build -t $(IMAGE) -f $(SERVICE)/Dockerfile . build-all: - $(foreach service,leecher processor transmitter, docker build --network=host -t ynput/ayon-shotgrid-$(service):$(ADDON_VERSION) -f $(service)/Dockerfile . &) + docker build -t ynput/ayon-shotgrid-leecher:$(ADDON_VERSION) -f ./leecher/Dockerfile . + docker build -t ynput/ayon-shotgrid-processor:$(ADDON_VERSION) -f ./processor/Dockerfile . + docker build -t ynput/ayon-shotgrid-transmitter:$(ADDON_VERSION) -f ./transmitter/Dockerfile . + +dist: + docker push $(IMAGE) + +dist-all: + docker push ynput/ayon-shotgrid-leecher:$(ADDON_VERSION) + docker push ynput/ayon-shotgrid-processor:$(ADDON_VERSION) + docker push ynput/ayon-shotgrid-transmitter:$(ADDON_VERSION) clean: if docker images "$(IMAGE)"; then \ @@ -34,10 +46,12 @@ clean: fi clean-build: clean - docker build --network=host --no-cache -t $(IMAGE) -f $(SERVICE)/Dockerfile . + docker build --no-cache -t $(IMAGE) -f $(SERVICE)/Dockerfile . clean-build-all: - $(foreach service,leecher processor transmitter, docker build --network=host --no-cache -t ynput/ayon-shotgrid-$(service):$(ADDON_VERSION) -f $(service)/Dockerfile . &) + docker build --no-cache -t ynput/ayon-shotgrid-leecher:$(ADDON_VERSION) -f ./leecher/Dockerfile . + docker build --no-cache -t ynput/ayon-shotgrid-processor:$(ADDON_VERSION) -f ./processor/Dockerfile . + docker build --no-cache -t ynput/ayon-shotgrid-transmitter:$(ADDON_VERSION) -f ./transmitter/Dockerfile . dev: @@ -51,7 +65,6 @@ dev: --attach=stdin \ --attach=stdout \ --attach=stderr \ - --network=host \ $(IMAGE) python -m $(SERVICE) find $(CURDIR)/$(SERVICE) -type l -delete diff --git a/services/leecher/README.md b/services/leecher/README.md index 74cc56fe..958c95e5 100644 --- a/services/leecher/README.md +++ b/services/leecher/README.md @@ -1,6 +1,6 @@ # Shotgrid Leecher -The Shotgrid leecher depends on the [Shotgrid Addon for Ayon](https://github.com/ynput/ayon-shotgrid-addon) since it expects certain settings provided once the addon is installed in the server. +The Shotgrid leecher depends on the [Shotgrid Addon for AYON](https://github.com/ynput/ayon-shotgrid-addon) since it expects certain settings provided once the addon is installed in the server. To get started, create a copy of the `sample_env` file and rename it to `.env` and set the values acordingly, after that you can run the service by issuing: ```sh diff --git a/services/leecher/leecher/listener.py b/services/leecher/leecher/listener.py index dd5a300b..9d213294 100644 --- a/services/leecher/leecher/listener.py +++ b/services/leecher/leecher/listener.py @@ -1,8 +1,8 @@ """ -A Shotgrid Events listener leecher for Ayon. +A Shotgrid Events listener leecher for AYON. This service will continually run and query the EventLogEntry table from -Shotgrid and converts them to Ayon events, and can be configured from the Ayon +Shotgrid and converts them to AYON events, and can be configured from the AYON Addon settings page. """ import sys @@ -45,7 +45,7 @@ class ShotgridListener: log = get_logger(__file__) def __init__(self): - """Ensure both Ayon and Shotgrid connections are available. + """Ensure both AYON and Shotgrid connections are available. Set up common needed attributes and handle shotgrid connection closure via signal handlers. @@ -232,7 +232,7 @@ def start_listening(self): """Main loop querying the Shotgrid database for new events Since Shotgrid does not have an event hub per se, we need to query - the "EventLogEntry table and send these as Ayon events for processing. + the "EventLogEntry table and send these as AYON events for processing. We try to continue from the last Event processed by the leecher, if none is found we start at the moment in time. @@ -293,7 +293,7 @@ def start_listening(self): if not event: continue - ignore_event = False + ignore_event = True last_event_id = event["id"] if ( @@ -349,7 +349,7 @@ def _is_api_user_event(self, event: dict[str, Any]) -> bool: def send_shotgrid_event_to_ayon( self, payload: dict[str, Any], sg_projects_by_id: dict[str, Any] ): - """Send the Shotgrid event as an Ayon event. + """Send the Shotgrid event as an AYON event. Args: payload (dict): The Event data. @@ -400,7 +400,7 @@ def send_shotgrid_event_to_ayon( }, ) - self.log.info("Dispatched Ayon event with payload:", payload) + self.log.info("Dispatched AYON event with payload:", payload) def service_main(): diff --git a/services/leecher/pyproject.toml b/services/leecher/pyproject.toml index b29d7636..c9154d75 100644 --- a/services/leecher/pyproject.toml +++ b/services/leecher/pyproject.toml @@ -1,7 +1,7 @@ [tool.poetry] name = "shotgrid-leecher" -version = "0.4.5-dev.1" -description = "Shotgrid Integration for Ayon" +version = "0.4.6+dev" +description = "Shotgrid Integration for AYON" authors = ["Oscar Domingo "] [tool.poetry.dependencies] diff --git a/services/manage.ps1 b/services/manage.ps1 index 3b51696a..038fd051 100644 --- a/services/manage.ps1 +++ b/services/manage.ps1 @@ -34,7 +34,7 @@ function Get-ServiceImage { # Show help message with details on how to use this script function Show-Help { Write-Host "" - Write-Host "Ayon Shotgrid $AddonVersion Service Builder" + Write-Host "AYON Shotgrid $AddonVersion Service Builder" Write-Host "" Write-Host "Usage: .\manage.ps1 [target] -Service [service-name]" Write-Host "" diff --git a/services/processor/processor/processor.py b/services/processor/processor/processor.py index e23602d8..a3148577 100644 --- a/services/processor/processor/processor.py +++ b/services/processor/processor/processor.py @@ -1,7 +1,7 @@ """ -A Shotgrid Events listener processor for Ayon. +A Shotgrid Events listener processor for AYON. -This service will continually run and query the Ayon Events Server in orther to +This service will continually run and query the AYON Events Server in orther to entroll the events of topic `shotgrid.leech` to perform processing of Shotgrid related events. """ diff --git a/services/processor/pyproject.toml b/services/processor/pyproject.toml index d81f71ff..0028b368 100644 --- a/services/processor/pyproject.toml +++ b/services/processor/pyproject.toml @@ -1,7 +1,7 @@ [tool.poetry] name = "shotgrid-processor" -version = "0.4.5-dev.1" -description = "Shotgrid Integration for Ayon" +version = "0.4.6+dev" +description = "Shotgrid Integration for AYON" authors = ["Oscar Domingo "] [tool.poetry.dependencies] diff --git a/services/shotgrid_common/ayon_shotgrid_hub/__init__.py b/services/shotgrid_common/ayon_shotgrid_hub/__init__.py index 680d6dab..d18f8c80 100644 --- a/services/shotgrid_common/ayon_shotgrid_hub/__init__.py +++ b/services/shotgrid_common/ayon_shotgrid_hub/__init__.py @@ -1,6 +1,6 @@ """ Influenced by the `ayon_api.EntityHub` the `AyonShotgridHub` is a class that provided a valid Project name and code, will perform all the necessary -checks and provide methods to keep an Ayon and Shotgrid project in sync. +checks and provide methods to keep an AYON and Shotgrid project in sync. """ import re @@ -139,7 +139,7 @@ def project_name(self, project_name): try: self._ay_project = EntityHub(project_name) self._ay_project.project_entity - except Exception as err: + except Exception: self.log.warning(f"Project {project_name} does not exist in AYON.") self._ay_project = None @@ -156,7 +156,7 @@ def project_name(self, project_name): self.project_name, custom_fields=custom_fields ) - except Exception as e: + except Exception: self.log.warning(f"Project {project_name} does not exist in Shotgrid. ") self._sg_project = None @@ -370,7 +370,9 @@ def react_to_ayon_event(self, ayon_event): the change encompases, i.e. a new shot, new asset, etc. """ if not self._sg_project[CUST_FIELD_CODE_AUTO_SYNC]: - self.log.info(f"Ignoring event, Shotgirid field 'Ayon Auto Sync' is disabled.") + self.log.info( + "Ignoring event, Shotgrid field 'Ayon Auto Sync' is disabled." + ) return match ayon_event["topic"]: @@ -400,7 +402,7 @@ def react_to_ayon_event(self, ayon_event): attrib_key = next(iter(ayon_event["payload"]["newValue"])) if attrib_key not in self.custom_attribs_map: self.log.warning( - f"Updating attribute '{attrib_key}' from Ayon to SG " + f"Updating attribute '{attrib_key}' from AYON to SG " f"not supported: {self.custom_attribs_map}." ) return diff --git a/services/shotgrid_common/ayon_shotgrid_hub/match_ayon_hierarchy_in_shotgrid.py b/services/shotgrid_common/ayon_shotgrid_hub/match_ayon_hierarchy_in_shotgrid.py index ee691aa0..03cc55bd 100644 --- a/services/shotgrid_common/ayon_shotgrid_hub/match_ayon_hierarchy_in_shotgrid.py +++ b/services/shotgrid_common/ayon_shotgrid_hub/match_ayon_hierarchy_in_shotgrid.py @@ -385,7 +385,7 @@ def _create_new_entity( ): data[sg_parent_field] = sg_parent_entity - # Fill up data with any extra attributes from Ayon we want to sync to SG + # Fill up data with any extra attributes from AYON we want to sync to SG data |= get_sg_custom_attributes_data( sg_session, ay_entity.attribs.to_dict(), diff --git a/services/shotgrid_common/ayon_shotgrid_hub/match_shotgrid_hierarchy_in_ayon.py b/services/shotgrid_common/ayon_shotgrid_hub/match_shotgrid_hierarchy_in_ayon.py index e9af92a5..6ae4c08b 100644 --- a/services/shotgrid_common/ayon_shotgrid_hub/match_shotgrid_hierarchy_in_ayon.py +++ b/services/shotgrid_common/ayon_shotgrid_hub/match_shotgrid_hierarchy_in_ayon.py @@ -166,7 +166,7 @@ def match_shotgrid_hierarchy_in_ayon( CUST_FIELD_CODE_ID: ay_entity.id, CUST_FIELD_CODE_SYNC: sg_entity_sync_status } - # Update Shotgrid entity with Ayon ID and sync status + # Update Shotgrid entity with AYON ID and sync status sg_session.update( sg_ay_dict["attribs"][SHOTGRID_TYPE_ATTRIB], sg_entity_id, @@ -178,16 +178,6 @@ def match_shotgrid_hierarchy_in_ayon( for sg_child_id in sg_ay_dicts_parents.get(sg_entity_id, []): sg_ay_dicts_deck.append((ay_entity, sg_child_id)) - try: - entity_hub.commit_changes() - except Exception: - log.error( - "Unable to commit all entities to AYON!", exc_info=True) - - log.info( - "Processed entities successfully!. " - f"Amount of entities: {len(processed_ids)}" - ) # Sync project attributes from Shotgrid to AYON entity_hub.project_entity.attribs.set( SHOTGRID_ID_ATTRIB, @@ -214,9 +204,18 @@ def match_shotgrid_hierarchy_in_ayon( attrib_value ) - entity_hub.commit_changes() + try: + entity_hub.commit_changes() + except Exception: + log.error( + "Unable to commit all entities to AYON!", exc_info=True) + + log.info( + "Processed entities successfully!. " + f"Amount of entities: {len(processed_ids)}" + ) - # Update Shotgrid project with Ayon ID and sync status + # Update Shotgrid project with AYON ID and sync status sg_session.update( "Project", sg_project["id"], @@ -243,8 +242,8 @@ def _create_new_entity( Args: entity_hub (ayon_api.EntityHub): The project's entity hub. - parent_entity: Ayon parent entity. - sg_ay_dict (dict): Ayon ShotGrid entity to create. + parent_entity: AYON parent entity. + sg_ay_dict (dict): AYON ShotGrid entity to create. """ if sg_ay_dict["type"].lower() == "task": # only create if parent_entity type is not project @@ -282,7 +281,7 @@ def _create_new_entity( try: # INFO: it was causing error so trying to set status directly ay_entity.status = status - except ValueError as e: + except ValueError: # `ValueError: Status ip is not available on project.` # log.warning(f"Status sync not implemented: {e}") pass @@ -333,4 +332,4 @@ def _create_color() -> str: color = [random.randint(0, 255) for _ in range(3)] if sum(color) < 400: color = [255 - x for x in color] - return f'#{"".join([f"{x:02x}" for x in color])}' \ No newline at end of file + return f'#{"".join([f"{x:02x}" for x in color])}' diff --git a/services/shotgrid_common/ayon_shotgrid_hub/update_from_ayon.py b/services/shotgrid_common/ayon_shotgrid_hub/update_from_ayon.py index 56de6067..b4bc3eb1 100644 --- a/services/shotgrid_common/ayon_shotgrid_hub/update_from_ayon.py +++ b/services/shotgrid_common/ayon_shotgrid_hub/update_from_ayon.py @@ -17,9 +17,9 @@ get_sg_custom_attributes_data ) from constants import ( - CUST_FIELD_CODE_ID, # Shotgrid Field for the Ayon ID. - SHOTGRID_ID_ATTRIB, # Ayon Entity Attribute. - SHOTGRID_TYPE_ATTRIB, # Ayon Entity Attribute. + CUST_FIELD_CODE_ID, # Shotgrid Field for the AYON ID. + SHOTGRID_ID_ATTRIB, # AYON Entity Attribute. + SHOTGRID_TYPE_ATTRIB, # AYON Entity Attribute. ) from utils import get_logger @@ -45,7 +45,7 @@ def create_sg_entity_from_ayon_event( sg_project (dict): The Shotgrid project. sg_enabled_entities (list): List of Shotgrid entities to be enabled. custom_attribs_map (dict): Dictionary that maps a list of attribute names from - Ayon to Shotgrid. + AYON to Shotgrid. Returns: ay_entity (ayon_api.entity_hub.EntityHub.Entity): The newly @@ -289,7 +289,7 @@ def remove_sg_entity_from_ayon_event( if not sg_entity: log.warning( - f"Unable to find Ayon entity with id '{ay_id}' in Shotgrid.") + f"Unable to find AYON entity with id '{ay_id}' in Shotgrid.") return sg_id = sg_entity["id"] @@ -421,7 +421,7 @@ def _create_sg_entity( CUST_FIELD_CODE_ID: ay_entity.id, } - # Fill up data with any extra attributes from Ayon we want to sync to SG + # Fill up data with any extra attributes from AYON we want to sync to SG data.update(get_sg_custom_attributes_data( sg_session, ay_entity.attribs.to_dict(), diff --git a/services/shotgrid_common/ayon_shotgrid_hub/update_from_shotgrid.py b/services/shotgrid_common/ayon_shotgrid_hub/update_from_shotgrid.py index e342debb..f51e728b 100644 --- a/services/shotgrid_common/ayon_shotgrid_hub/update_from_shotgrid.py +++ b/services/shotgrid_common/ayon_shotgrid_hub/update_from_shotgrid.py @@ -12,7 +12,7 @@ "retirement_date": "2023-03-31 15:26:16 UTC" } -And most of the times it fetches the ShotGrid entity as an Ayon dict like: +And most of the times it fetches the ShotGrid entity as an AYON dict like: { "label": label, "name": name, @@ -34,9 +34,9 @@ update_ay_entity_custom_attributes, ) from constants import ( - CUST_FIELD_CODE_ID, # ShotGrid Field for the Ayon ID. - SHOTGRID_ID_ATTRIB, # Ayon Entity Attribute. - SHOTGRID_TYPE_ATTRIB, # Ayon Entity Attribute. + CUST_FIELD_CODE_ID, # ShotGrid Field for the AYON ID. + SHOTGRID_ID_ATTRIB, # AYON Entity Attribute. + SHOTGRID_TYPE_ATTRIB, # AYON Entity Attribute. SHOTGRID_REMOVED_VALUE, # Value for removed entities. SG_RESTRICTED_ATTR_FIELDS, ) @@ -66,7 +66,7 @@ def create_ay_entity_from_sg_event( sg_enabled_entities (list[str]): List of entity strings enabled. project_code_field (str): The Shotgrid project code field. custom_attribs_map (Optional[dict]): A dictionary that maps ShotGrid - attributes to Ayon attributes. + attributes to AyonAYON attributes. Returns: ay_entity (ayon_api.entity_hub.EntityHub.Entity): The newly @@ -109,10 +109,10 @@ def create_ay_entity_from_sg_event( if ay_entity: log.debug("ShotGrid Entity exists in AYON.") - # Ensure Ayon Entity has the correct ShotGrid ID + # Ensure AYON Entity has the correct ShotGrid ID ayon_entity_sg_id = str( ay_entity.attribs.get_attribute(SHOTGRID_ID_ATTRIB).value) - # Ensure Ayon Entity has the correct Shotgrid ID + # Ensure AYON Entity has the correct Shotgrid ID ay_shotgrid_id = str( sg_ay_dict["attribs"].get(SHOTGRID_ID_ATTRIB, "")) if ayon_entity_sg_id != ay_shotgrid_id: @@ -173,7 +173,7 @@ def create_ay_entity_from_sg_event( # This really should be an edge ase, since any parent event would # happen before this... but hey raise ValueError( - "Parent does not exist in Ayon, try doing a Project Sync.") + "Parent does not exist in AYON, try doing a Project Sync.") if sg_ay_dict["type"].lower() == "task": ay_entity = ayon_entity_hub.add_new_task( @@ -229,7 +229,7 @@ def update_ayon_entity_from_sg_event( project_code_field: str, custom_attribs_map: Optional[Dict[str, str]] = None ): - """Try to update an entity in Ayon. + """Try to update an entity in AYON. Args: sg_event (dict): The `meta` key from a ShotGrid Event. @@ -239,7 +239,7 @@ def update_ayon_entity_from_sg_event( sg_enabled_entities (list[str]): List of entity strings enabled. project_code_field (str): The ShotGrid project code field. custom_attribs_map (dict): A dictionary that maps ShotGrid - attributes to Ayon attributes. + attributes to AYON attributes. Returns: ay_entity (ayon_api.entity_hub.EntityHub.Entity): The modified entity. @@ -260,7 +260,7 @@ def update_ayon_entity_from_sg_event( ) return - # if the entity does not have an Ayon ID, try to create it + # if the entity does not have an AYON ID, try to create it # and no need to update if not sg_ay_dict["data"].get(CUST_FIELD_CODE_ID): log.debug(f"Creating AYON Entity: {sg_ay_dict}") @@ -293,13 +293,13 @@ def update_ayon_entity_from_sg_event( ): raise ValueError("Entity is immutable, aborting...") - # Ensure Ayon Entity has the correct ShotGrid ID + # Ensure AYON Entity has the correct ShotGrid ID ayon_entity_sg_id = str( ay_entity.attribs.get_attribute(SHOTGRID_ID_ATTRIB).value) sg_entity_sg_id = str( sg_ay_dict["attribs"].get(SHOTGRID_ID_ATTRIB, "") ) - log.debug(f"Updating Ayon Entity: {ay_entity.name}") + log.debug(f"Updating AYON Entity: {ay_entity.name}") if ayon_entity_sg_id != sg_entity_sg_id: raise ValueError("Mismatching ShotGrid IDs, aborting...") @@ -332,13 +332,13 @@ def remove_ayon_entity_from_sg_event( ayon_entity_hub: ayon_api.entity_hub.EntityHub, project_code_field: str, ): - """Try to remove an entity in Ayon. + """Try to remove an entity in AYON. Args: sg_event (dict): The `meta` key from a ShotGrid Event. sg_session (shotgun_api3.Shotgun): The ShotGrid API session. ayon_entity_hub (ayon_api.entity_hub.EntityHub): The AYON EntityHub. - project_code_field (str): The ShotGrid field that contains the Ayon ID. + project_code_field (str): The ShotGrid field that contains the AYON ID. """ # for now we are ignoring Task type entities # TODO: Handle Task entities @@ -376,7 +376,7 @@ def remove_ayon_entity_from_sg_event( if not sg_ay_dict["data"].get(CUST_FIELD_CODE_ID): log.warning( - "Entity does not have an Ayon ID, aborting..." + "Entity does not have an AYON ID, aborting..." ) return diff --git a/services/shotgrid_common/utils.py b/services/shotgrid_common/utils.py index 7cb4a3b1..db2d4c15 100644 --- a/services/shotgrid_common/utils.py +++ b/services/shotgrid_common/utils.py @@ -97,8 +97,8 @@ def _sg_to_ay_dict( ) -> dict: """Morph a ShotGrid entity dict into an ayon-api Entity Hub compatible one. - Create a dictionary that follows the Ayon Entity Hub schema and handle edge - cases so it's ready for Ayon consumption. + Create a dictionary that follows the AYON Entity Hub schema and handle edge + cases so it's ready for AYON consumption. Folders: https://github.com/ynput/ayon-python-api/blob/30d702618b58676c3708f09f131a0974a92e1002/ayon_api/entity_hub.py#L2397 # noqa Tasks: https://github.com/ynput/ayon-python-api/blob/30d702618b58676c3708f09f131a0974a92e1002/ayon_api/entity_hub.py#L2579 # noqa @@ -149,7 +149,7 @@ def _sg_to_ay_dict( "data": { # We store the ShotGrid ID and the Sync status in the data # dictionary so we can easily access them when needed - # And avoid any conflicts with the Ayon attributes we only set + # And avoid any conflicts with the AYON attributes we only set # sync status to "Failed" if the ID is not set CUST_FIELD_CODE_SYNC: ( sg_entity.get(CUST_FIELD_CODE_SYNC) @@ -183,7 +183,7 @@ def create_ay_fields_in_sg_entities( custom_attribs_map: dict, custom_attribs_types: dict ): - """Create Ayon fields in ShotGrid entities. + """Create AYON fields in ShotGrid entities. Some fields need to exist in the ShotGrid Entities, mainly the `sg_ayon_id` and `sg_ayon_sync_status` for the correct operation of the handlers. @@ -213,7 +213,7 @@ def create_ay_fields_in_sg_entities( CUST_FIELD_CODE_SYNC, field_properties={ "name": "Ayon Sync Status", - "description": "The Synchronization status with Ayon.", + "description": "The Synchronization status with AYON.", "valid_values": ["Synced", "Failed", "Skipped"], } ) @@ -233,7 +233,7 @@ def create_ay_custom_attribs_in_sg_entity( custom_attribs_map: dict, custom_attribs_types: dict ): - """Create Ayon custom attributes in ShotGrid entities. + """Create AYON custom attributes in ShotGrid entities. Args: sg_session (shotgun_api3.Shotgun): Instance of a ShotGrid API Session. @@ -282,7 +282,7 @@ def create_ay_fields_in_sg_project( custom_attribs_map: dict, custom_attribs_types: dict ): - """Create Ayon Project fields in ShotGrid. + """Create AYON Project fields in ShotGrid. This will create Project Unique attributes into ShotGrid. @@ -343,7 +343,7 @@ def create_sg_entities_in_ay( sg_enabled_entities (list): The enabled entities. """ - # Types of SG entities to ignore as Ayon folders + # Types of SG entities to ignore as AYON folders ignored_folder_types = {"task", "version"} # Find ShotGrid Entities that are to be treated as folders @@ -403,7 +403,7 @@ def create_asset_category(entity_hub, parent_entity, sg_ay_dict): Args: entity_hub (ayon_api.EntityHub): The project's entity hub. parent_entity: AYON parent entity. - sg_ay_dict (dict): The ShotGrid entity ready for Ayon consumption. + sg_ay_dict (dict): The ShotGrid entity ready for AYON consumption. """ asset_category = sg_ay_dict["data"]["sg_asset_type"] # asset category entity name @@ -440,8 +440,8 @@ def get_asset_category(entity_hub, parent_entity, sg_ay_dict): Args: entity_hub (ayon_api.EntityHub): The project's entity hub. - parent_entity: Ayon parent entity. - sg_ay_dict (dict): The ShotGrid entity ready for Ayon consumption. + parent_entity: AYON parent entity. + sg_ay_dict (dict): The ShotGrid entity ready for AYON consumption. """ # just in case the asset type doesn't exist yet @@ -694,7 +694,7 @@ def get_sg_entity_as_ay_dict( extra_fields: Optional[list] = None, retired_only: Optional[bool] = False, ) -> dict: - """Get a ShotGrid entity, and morph it to an Ayon compatible one. + """Get a ShotGrid entity, and morph it to an AYON compatible one. Args: sg_session (shotgun_api3.Shotgun): Shotgun Session object. @@ -706,7 +706,7 @@ def get_sg_entity_as_ay_dict( extra_fields (Optional[list]): List of optional fields to query. retired_only (bool): Whether to return only retired entities. Returns: - new_entity (dict): The ShotGrid entity ready for Ayon consumption. + new_entity (dict): The ShotGrid entity ready for AYON consumption. """ query_fields = list(SG_COMMON_ENTITY_FIELDS) if extra_fields and isinstance(extra_fields, list): @@ -784,7 +784,7 @@ def get_sg_entity_parent_field( def get_sg_missing_ay_attributes(sg_session: shotgun_api3.Shotgun): - """ Ensure all the Ayon required fields are present in ShotGrid. + """ Ensure all the AYON required fields are present in ShotGrid. Args: sg_session (shotgun_api3.Shotgun): Instance of a ShotGrid API Session. @@ -953,7 +953,7 @@ def get_sg_statuses( """ # If given an entity type, we filter out the statuses by just the ones # supported by that entity - # NOTE: this is a limitation in Ayon as the statuses are global and not + # NOTE: this is a limitation in AYON as the statuses are global and not # per entity if sg_entity_type: if sg_entity_type == "Project": @@ -1075,7 +1075,7 @@ def update_ay_entity_custom_attributes( custom_attribs_map: dict, values_to_update: Optional[list] = None, ): - """Update Ayon entity custom attributes from ShotGrid dictionary""" + """Update AYON entity custom attributes from ShotGrid dictionary""" for ay_attrib, _ in custom_attribs_map.items(): if values_to_update and ay_attrib not in values_to_update: continue diff --git a/services/transmitter/pyproject.toml b/services/transmitter/pyproject.toml index 16ad0f66..6878a8d0 100644 --- a/services/transmitter/pyproject.toml +++ b/services/transmitter/pyproject.toml @@ -1,7 +1,7 @@ [tool.poetry] name = "shotgrid-transmitter" -version = "0.4.5-dev.1" -description = "Shotgrid Integration for Ayon" +version = "0.4.6+dev" +description = "Shotgrid Integration for AYON" authors = ["Oscar Domingo "] [tool.poetry.dependencies] diff --git a/services/transmitter/transmitter/transmitter.py b/services/transmitter/transmitter/transmitter.py index bdef3f7a..a4329771 100644 --- a/services/transmitter/transmitter/transmitter.py +++ b/services/transmitter/transmitter/transmitter.py @@ -1,7 +1,7 @@ """ A AYON Events listener to push changes to Shotgrid. -This service will continually run and query the Ayon Events Server in order to +This service will continually run and query the AYON Events Server in order to enroll the events of topic `entity.folder` and `entity.task` when any of the two are `created`, `renamed` or `deleted`. """ @@ -23,7 +23,7 @@ class ShotgridTransmitter: _sg: shotgun_api3.Shotgun = None def __init__(self): - """ Ensure both Ayon and Shotgrid connections are available. + """ Ensure both AYON and Shotgrid connections are available. Set up common needed attributes and handle shotgrid connection closure via signal handlers.