feat(examples): Added examples for Spot Event Plugin Deployment (#180)

Adds python and TS examples that create a farm that has the requirements to use
Deadline's Spot Event Plugin.

examples/deadline/All-In-AWS-Infrastructure-Basic/python/package/lib/storage_tier.py

@@ -126,7 +126,9 @@ def __init__(self, scope: Construct, stack_id: str, *, props: StorageTierDocDBPr
             self,
             'DocDBCluster',
             instance_props=instance_props,
-            instances=len(self.availability_zones),
+            # TODO - For cost considerations this example only uses 1 Database instance. 
+            # It is recommended that when creating your render farm you use at least 2 instances for redundancy.
+            instances=1,
             master_user=Login(username='adminuser'),
             backup=BackupProps(
                 # We recommend setting the retention of your backups to 15 days

examples/deadline/All-In-AWS-Infrastructure-Basic/ts/lib/storage-tier.ts

@@ -110,7 +110,9 @@ export class StorageTierDocDB extends StorageTier {
         vpcSubnets: { subnetType: SubnetType.PRIVATE },
         instanceType: props.databaseInstanceType,
       },
-      instances: this.availabilityZones.length,
+      // TODO - For cost considerations this example only uses 1 Database instance. 
+      // It is recommended that when creating your render farm you use at least 2 instances for redundancy.
+      instances: 1,
       masterUser: {
         username: 'adminuser',
       },

examples/deadline/All-In-AWS-Infrastructure-SEP/README.md

@@ -0,0 +1,46 @@
+# RFDK Sample Application - Deadline Spot Event Plugin
+
+This is a sample RFDK application that deploys the basic infrastructure for a Deadline render farm that uses Deadline's Spot Event Plugin for auto scaling workers.
+
+---
+
+_**Note:** This application is an illustrative example to showcase some of the capabilities of the RFDK. **It is not intended to be used for production render farms**, which should be built with more consideration of the security and operational needs of the system._
+
+---
+
+## Architecture
+
+This sample application deploys a basic Deadline Render farm that is configured to use Deadlines [Spot Event Plugin](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html). 
+
+### Components
+
+All components in the render farm live within a [VPC](https://aws.amazon.com/vpc/).
+
+#### Repository
+
+The Repository component contains the database and file system that store persistent data used by Deadline. These resources are initialized by the Deadline Repository installer. The database in this example uses [Amazon DocumentDB](https://aws.amazon.com/documentdb/).
+
+#### Render Queue
+
+The Render Queue component contains the fleet of [Deadline Remote Connection Server](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/remote-connection-server.html) instances behind an [Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html). This acts as the central service for Deadline applications and is the only component that interacts with the Repository.  When comparing this component to the "All in AWS Infrastructure - Basic" example it has been granted additional permissions in order to use the Spot Event Plugin.    
+
+#### Spot Event Plugin Configurations
+
+The Spot Event plugin requires additional Roles for both Deadline's Resource Tracker and the Spot Workers that are created and a Security Group to allow your Spot workers the ability to access the Render Queue.
+
+## Prerequisites
+
+- The Spot Fleet Configuration requires an [Amazon Machine Image (AMI)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) with the Deadline Worker application installed. This AMI must have Deadline Installed and should be configured to connect to your repository.  For additional information on setting up your AMI please see the [Spot Event Plugin Documentation](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html).
+- You have setup and configured the AWS CLI
+- Your AWS account already has CDK bootstrapped in the desired region by running `cdk bootstrap`
+- You must have NodeJS installed on your system
+- You must have Docker installed on your system
+- You must have Python 3.7+ installed on your system (Python app only)
+
+## Typescript
+
+[Continue to Typescript specific documentation.](ts/README.md)
+
+## Python
+
+[Continue to Python specific documentation.](python/README.md)

examples/deadline/All-In-AWS-Infrastructure-SEP/python/.gitignore

@@ -0,0 +1,12 @@
+*.swp
+package-lock.json
+__pycache__
+.pytest_cache
+.env
+*.egg-info
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out
+cdk.context.json
+stage

examples/deadline/All-In-AWS-Infrastructure-SEP/python/README.md

@@ -0,0 +1,65 @@
+# RFDK Sample Application - Deadline Spot Event Plugin - Python
+
+## Overview
+[Back to overview](../README.md)
+
+## Instructions
+
+---
+**NOTE**
+
+These instructions assume that your working directory is `examples/deadline/All-In-AWS-Infrastructure-SEP/python/` relative to the root of the AWS-RFDK package.
+
+---
+
+1.  This sample app on the `mainline` branch may contain features that have not yet been officially released, and may not be available in the `aws-rfdk` package installed through pip from PyPI. To work from an example of the latest release, please switch to the `release` branch. If you would like to try out unreleased features, you can stay on `mainline` and follow the instructions for building, packing, and installing the `aws-rfdk` from your local repository.
+2.  Install the dependencies of the sample app:
+
+    ```bash
+    pip install -r requirements.txt
+    ```
+3.  If working on the `release` branch, this step can be skipped. If working on `mainline`, navigate to the base directory where the build and packaging scripts are, then run them and install the result over top of the `aws-rfdk` version that was installed in the previous step:
+    ```bash
+    # Navigate to the root directory of the RFDK repository
+    pushd ../../../..
+    # Enter the Docker container to run the build and pack scripts
+    ./scripts/rfdk_build_environment.sh
+    ./build.sh
+    ./pack.sh
+    # Exit the Docker container
+    exit
+    # Navigate back to the example directory
+    popd
+    pip install ../../../../dist/python/aws-rfdk-<version>.tar.gz
+    ```
+3. Stage the Docker recipes for `RenderQueue`:
+
+    ```bash
+    # Set this value to the version of RFDK your application targets
+    RFDK_VERSION=<version_of_RFDK>
+
+    # Set this value to the version of AWS Thinkbox Deadline you'd like to deploy to your farm. Deadline 10.1.9 and up are supported.
+    RFDK_DEADLINE_VERSION=<version_of_deadline>
+
+    npx --package=aws-rfdk@${RFDK_VERSION} stage-deadline --output stage ${RFDK_DEADLINE_VERSION}
+    ```
+4. Deploy all the stacks in the sample app:
+
+    ```bash
+    cdk deploy "*"
+    ```
+
+5. Connect to your Render Farm and open up the Deadline Monitor.
+
+6. Configure the Spot event plugin by following the directions in the [Spot Event Plugin documentation](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html) with the following considerations:
+
+    Use the default security credentials by using turning "Use Local Credentials" to False and leaving both "Access Key ID" and "Secret Access Key" blank.
+    Ensure that the Region your Spot workers will be launched in is the same region as your CDK application.
+    When Creating your Spot Fleet Requests, set the IAM instance profile to "DeadlineSpotWorkerRole" and set the security group to "DeadlineSpotSecurityGroup".
+    Configure your instances to connect to the Render Queue by either creating your AMI after launching your app and preconfiguring the AMI or by setting up a userdata in the Spot Fleet Request. (see the Spot Event Plugin documentation for additional information on configuring this connection.)
+
+7. Once you are finished with the sample app, you can tear it down by running:
+
+    ```bash
+    cdk destroy "*"
+    ```

examples/deadline/All-In-AWS-Infrastructure-SEP/python/cdk.json

@@ -0,0 +1,3 @@
+{
+  "app": "python -m package.app"
+}

examples/deadline/All-In-AWS-Infrastructure-SEP/python/package/app.py

@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+
+# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+import os
+
+from aws_cdk.core import (
+    App,
+    Environment
+)
+
+from .lib import (
+    sep_stack,
+)
+
+def main():
+    # ------------------------------
+    # Application
+    # ------------------------------
+    app = App()
+
+    if 'CDK_DEPLOY_ACCOUNT' not in os.environ and 'CDK_DEFAULT_ACCOUNT' not in os.environ:
+        raise ValueError('You must define either CDK_DEPLOY_ACCOUNT or CDK_DEFAULT_ACCOUNT in the environment.')
+    if 'CDK_DEPLOY_REGION' not in os.environ and 'CDK_DEFAULT_REGION' not in os.environ:
+        raise ValueError('You must define either CDK_DEPLOY_REGION or CDK_DEFAULT_REGION in the environment.')
+    env = Environment(
+        account=os.environ.get('CDK_DEPLOY_ACCOUNT', os.environ.get('CDK_DEFAULT_ACCOUNT')),
+        region=os.environ.get('CDK_DEPLOY_REGION', os.environ.get('CDK_DEFAULT_REGION'))
+    )
+    # ------------------------------
+    # Service Tier
+    # ------------------------------
+    sep_props = sep_stack.SEPStackProps(
+        docker_recipes_stage_path=os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir, 'stage'),
+    )
+    service = sep_stack.SEPStack(app, 'SEPStack', props=sep_props, env=env)
+
+    app.synth()
+
+
+if __name__ == '__main__':
+    main()

examples/deadline/All-In-AWS-Infrastructure-SEP/python/package/lib/sep_stack.py

@@ -0,0 +1,126 @@
+# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+import typing
+from dataclasses import dataclass
+
+from aws_cdk.core import (
+    Construct,
+    Duration,
+    Stack,
+    StackProps
+)
+from aws_cdk.aws_ec2 import (
+    SecurityGroup,
+    Vpc,
+)
+from aws_cdk.aws_iam import (
+    ManagedPolicy,
+    Role,
+    ServicePrincipal 
+)
+from aws_rfdk.deadline import (
+    RenderQueue,
+    Repository,
+    Stage,
+    ThinkboxDockerRecipes,
+)
+
+
+@dataclass
+class SEPStackProps(StackProps):
+    """
+    Properties for ServiceTier
+    """
+    # The path to the directory where the staged Deadline Docker recipes are.
+    docker_recipes_stage_path: str
+
+
+class SEPStack(Stack):
+    """
+    The service tier contains all "business-logic" constructs
+    (e.g. Render Queue, UBL Licensing/License Forwarder, etc.).
+    """
+
+    def __init__(self, scope: Construct, stack_id: str, *, props: SEPStackProps, **kwargs):
+        """
+        Initialize a new instance of ServiceTier
+        :param scope: The scope of this construct.
+        :param stack_id: The ID of this construct.
+        :param props: The properties for this construct.
+        :param kwargs: Any kwargs that need to be passed on to the parent class.
+        """
+        super().__init__(scope, stack_id, **kwargs)
+
+         # The VPC that all components of the render farm will be created in.
+        vpc = Vpc(
+            self,
+            'Vpc',
+            max_azs=2
+        )
+
+        recipes = ThinkboxDockerRecipes(
+            self,
+            'Image',
+            stage=Stage.from_directory(props.docker_recipes_stage_path)
+        )
+
+        repository = Repository(
+            self,
+            'Repository',
+            vpc=vpc,
+            version=recipes.version,
+            repository_installation_timeout=Duration.minutes(20)
+        )
+
+        render_queue = RenderQueue(
+            self,
+            'RenderQueue',
+            vpc=props.vpc,
+            version=recipes.version,
+            images=recipes.render_queue_images,
+            repository=repository,
+            # TODO - Evaluate deletion protection for your own needs. This is set to false to
+            # cleanly remove everything when this stack is destroyed. If you would like to ensure
+            # that this resource is not accidentally deleted, you should set this to true.
+            deletion_protection=False
+        )
+        # Adds the following IAM managed Policies to the Render Queue so it has the necessary permissions
+        # to run the Spot Event Plugin and launch a Resource Tracker:
+        # * AWSThinkboxDeadlineSpotEventPluginAdminPolicy
+        # * AWSThinkboxDeadlineResourceTrackerAdminPolicy
+        render_queue.add_sep_policies()
+
+        # Create the security group that you will assign to your workers
+        worker_security_group = SecurityGroup(
+            self,
+            'SpotSecurityGroup', 
+            vpc=props.vpc,
+            allow_all_outbound=True,
+            security_group_name='DeadlineSpotSecurityGroup',
+        )
+        worker_security_group.connections.allow_to_default_port(
+            render_queue.endpoint
+        )
+
+        # Create the IAM Role for the Spot Event Plugins workers.
+        # Note: This Role MUST have a roleName that begins with "DeadlineSpot"
+        # Note: If you already have a worker IAM role in your account you can remove this code.
+        worker_iam_role = Role(
+            self,
+            'SpotWorkerRole',
+            assumed_by=ServicePrincipal('ec2.amazonaws.com'),
+            managed_policies= [ManagedPolicy.from_aws_managed_policy_name('AWSThinkboxDeadlineSpotEventPluginWorkerPolicy')],
+            role_name= 'DeadlineSpotWorkerRole',
+        )
+
+        # Creates the Resource Tracker Access role.  This role is required to exist in your account so the resource tracker will work properly
+        # Note: If you already have a Resource Tracker IAM role in your account you can remove this code.
+        Role(
+            self,
+            'ResourceTrackerRole',
+            assumed_by=ServicePrincipal('lambda.amazonaws.com'),
+            managed_policies= [ManagedPolicy.from_aws_managed_policy_name('AWSThinkboxDeadlineResourceTrackerAccessPolicy')],
+            role_name= 'DeadlineResourceTrackerAccessRole',
+        )
+

examples/deadline/All-In-AWS-Infrastructure-SEP/python/requirements.txt

@@ -0,0 +1 @@
+-e .

examples/deadline/All-In-AWS-Infrastructure-SEP/python/setup.py

@@ -0,0 +1,25 @@
+import setuptools
+
+
+with open("README.md") as fp:
+    long_description = fp.read()
+
+
+setuptools.setup(
+    name="all_in_aws_infrastructure_sep",
+    version="0.0.1",
+
+    description="RFDK All In AWS Infrastructure SEP",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+
+    package_dir={"": "package"},
+    packages=setuptools.find_packages(where="package"),
+
+    install_requires=[
+        "aws-cdk.core==1.72.0",
+        "aws-rfdk==0.20.0"
+    ],
+
+    python_requires=">=3.7",
+)

examples/deadline/All-In-AWS-Infrastructure-SEP/python/source.bat

@@ -0,0 +1,13 @@
+@echo off
+
+rem The sole purpose of this script is to make the command
+rem
+rem     source .env/bin/activate
+rem
+rem (which activates a Python virtualenv on Linux or Mac OS X) work on Windows.
+rem On Windows, this command just runs this batch file (the argument is ignored).
+rem
+rem Now we don't need to document a Windows command for activating a virtualenv.
+
+echo Executing .env\Scripts\activate.bat for you
+.env\Scripts\activate.bat

examples/deadline/All-In-AWS-Infrastructure-SEP/ts/.gitignore

@@ -0,0 +1,20 @@
+*.d.ts
+*.js
+cdk.out
+cdk.context.json
+
+# Used by nyc
+.nyc_output
+coverage
+.nycrc
+
+# Exclude package artifacts
+dist
+*.tgz
+.LAST_PACKAGE
+
+#Defines license that must be present
+!license-header.js
+
+# The staged files for Deadline
+stage