Merge pull request #622 from horsmand/bump/0.38.0

chore(release): 0.38.0

.github/workflows/ci.yml

@@ -5,9 +5,9 @@ name: CI
 
 on:
   push:
-    branches: [ mainline ]
+    branches: [ mainline, 'feature*' ]
   pull_request:
-    branches: [ mainline ]
+    branches: [ mainline, 'feature*' ]
 
 jobs:
   build:
@@ -23,7 +23,7 @@ jobs:
     steps:
     - uses: actions/checkout@v2.3.4
     - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v2.1.5
+      uses: actions/setup-node@v2.4.0
       with:
         node-version: ${{ matrix.node-version }}
     - run: npm install --global yarn

CHANGELOG.md

@@ -2,6 +2,36 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+## [0.38.0](https://github.com/aws/aws-rfdk/compare/v0.37.0...v0.38.0) (2021-10-25)
+
+
+### Supported CDK Version
+
+* [1.129.0](https://github.com/aws/aws-cdk/releases/tag/v1.129.0)
+
+
+### Officially Supported Deadline Versions
+
+* [10.1.9.2 to 10.1.19.4](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/release-notes.html)
+
+
+### Features
+
+* **deadline:** add Deadline Secrets Management integration in the Render Queue ([#528](https://github.com/aws/aws-rfdk/issues/528)) ([48baa18](https://github.com/aws/aws-rfdk/commit/48baa185b274030cab29a235469536585822313f))
+* **deadline:** add Secret Management support for Repository ([#514](https://github.com/aws/aws-rfdk/issues/514)) ([8c7dda6](https://github.com/aws/aws-rfdk/commit/8c7dda6deaa826e2efec379c9bf67b30fce02a89))
+* **deadline:** configure identity registration settings for deadline clients ([#576](https://github.com/aws/aws-rfdk/issues/576)) ([b9082b2](https://github.com/aws/aws-rfdk/commit/b9082b2014d3817c9eb9b3ecba1d2aaa54382074))
+* **deadline:** validate minimum Deadline version for secrets management ([#573](https://github.com/aws/aws-rfdk/issues/573)) ([6d5950e](https://github.com/aws/aws-rfdk/commit/6d5950e892d2a83ab11db247d33f8a5de22d360c))
+* **examples:** add deadline secrets management options to basic example app ([#562](https://github.com/aws/aws-rfdk/issues/562)) ([bd31a8d](https://github.com/aws/aws-rfdk/commit/bd31a8d6b748d6a4e242a0528addd42a71d2d55f))
+* **examples:** use dedicated subnets in All-In-AWS-Infrastructure-Basic example ([#598](https://github.com/aws/aws-rfdk/issues/598)) ([7aaec14](https://github.com/aws/aws-rfdk/commit/7aaec14db8fe8a9055d3672493d314b3d4127d09))
+
+
+### Bug Fixes
+
+* **deadline:** allow traffic from RenderQueue to UsageBasedLicensing ([#617](https://github.com/aws/aws-rfdk/issues/617)) ([dfbf88f](https://github.com/aws/aws-rfdk/commit/dfbf88f6478c30e0dec2d0939473f02268f669d9))
+* **deadline:** fix issue in client TLS configuration for Deadline 10.1.18 ([#543](https://github.com/aws/aws-rfdk/issues/543)) ([05b14f9](https://github.com/aws/aws-rfdk/commit/05b14f9ed5810d876c3a3df0293cb81531e833f5))
+* **deadline:** reinstall repository even if version is not changed ([821bab2](https://github.com/aws/aws-rfdk/commit/821bab291da27b226f74d4a9c5a01f1189cfb5e4))
+
+
 ## [0.37.0](https://github.com/aws/aws-rfdk/compare/v0.36.0...v0.37.0) (2021-08-05)
 
 
@@ -241,7 +271,7 @@ our examples for an illustration of the code update required.
 
 ### Security Notice
 
-RFDK version 0.27.x and later include security enhancements.  We recommend you upgrade RFDK and Deadline to further restrict the permissions required for RFDK & Deadline to function. Please upgrade the version of RFDK used in your CDK application to 0.27.x, and configure your application to deploy Deadline 10.1.14.x or later to resolve the issue. 
+RFDK version 0.27.x and later include security enhancements.  We recommend you upgrade RFDK and Deadline to further restrict the permissions required for RFDK & Deadline to function. Please upgrade the version of RFDK used in your CDK application to 0.27.x, and configure your application to deploy Deadline 10.1.14.x or later to resolve the issue.
 
 If you have an existing deployment that was built with RFDK versions 0.26.x or earlier, you will need to upgrade to RFDK 0.27.x and Deadline 10.1.14.x or later before June 10, 2021 @ 1:00PM PST/ 3:00PM CST/ 4:00PM EST. Failure to upgrade by the above date may result in disruptions to your render farm. If you have any questions, please contact AWS Thinkbox Customer Support at https://support.thinkboxsoftware.com/.
 

examples/deadline/All-In-AWS-Infrastructure-Basic/python/README.md

@@ -123,12 +123,36 @@ These instructions assume that your working directory is `examples/deadline/All-
     ```python
     self.alarm_email_address: Optional[str] = 'username@yourdomain.com'
     ```
-15. Deploy all the stacks in the sample app:
+15. Deadline Secrets Management is a feature used to encrypt certain values in the database that need to be kept secret. Additional documentation about the feature and how it works in the RFDK can be found in the [RFDK README](../../../../packages/aws-rfdk/lib/deadline/README.md). By default, Deadline Secrets Management is enabled, but it can be disabled by changing the `enable_secrets_management` variable in `package/config.py`.
+
+    ```python
+    self.enable_secrets_management: bool = False
+    ```
+
+16. When you are using Deadline Secrets Management you can define your own admin credentials by creating a Secret in AWS SecretsManager in the following format:
+
+    ```json
+        {
+            "username": "<admin user name>",
+            "password": "<admin user password>",
+        }
+    ```
+    The password must be at least 8 characters long and contain at least one lowercase, one uppercase, one digit, and one special character.
+
+    Then the value of the `secrets_management_secret_arn` variable in `package/config.py` should be changed to this Secret's ARN:
+
+    ```python
+    self.secrets_management_secret_arn: Optional[str] = '<your secret arn>'
+    ```
+    
+    It is highly recommended that you leave this parameter undefined to enable the automatic generation of a strong password.
+
+17. Deploy all the stacks in the sample app:
 
     ```bash
     cdk deploy "*"
     ```
-16. Once you are finished with the sample app, you can tear it down by running:
+18. Once you are finished with the sample app, you can tear it down by running:
 
     ```bash
     cdk destroy "*"

examples/deadline/All-In-AWS-Infrastructure-Basic/python/package/app.py

@@ -44,6 +44,9 @@ def main():
     if 'region' in config.deadline_client_linux_ami_map:
         raise ValueError('Deadline Client Linux AMI map is required but was not specified.')
 
+    if not config.enable_secrets_management and config.secrets_management_secret_arn:
+        print('Deadline Secrets Management is disabled, so the admin credentials specified in the provided secret will not be used.')
+
     # ------------------------------
     # Application
     # ------------------------------
@@ -110,7 +113,9 @@ def main():
         root_ca=security.root_ca,
         dns_zone=network.dns_zone,
         deadline_version=config.deadline_version,
-        accept_aws_thinkbox_eula=config.accept_aws_thinkbox_eula
+        accept_aws_thinkbox_eula=config.accept_aws_thinkbox_eula,
+        enable_secrets_management=config.enable_secrets_management,
+        secrets_management_secret_arn=config.secrets_management_secret_arn
     )
     service = service_tier.ServiceTier(app, 'ServiceTier', props=service_props, env=env)
 

examples/deadline/All-In-AWS-Infrastructure-Basic/python/package/config.py

@@ -40,10 +40,10 @@ def __init__(self):
         # to pin to. Some examples of pinned version values are "10", "10.1", or "10.1.12"
         self.deadline_version: Optional[str] = None
 
-        # A map of regions to Deadline Client Linux AMIs. As an example, the Linux Deadline 10.1.15.2 AMI ID
+        # A map of regions to Deadline Client Linux AMIs. As an example, the base Linux Deadline 10.1.19.4 AMI ID
         # from us-west-2 is filled in. It can be used as-is, added to, or replaced. Ideally the version here should match the version of
         # Deadline used in any connected Deadline constructs.
-        self.deadline_client_linux_ami_map: Mapping[str, str] = {'us-west-2': 'ami-0c8431fc72742c110'}
+        self.deadline_client_linux_ami_map: Mapping[str, str] = {'us-west-2': 'ami-04ae356533dc07fb5'}
 
         # A secret (in binary form) in SecretsManager that stores the UBL certificates in a .zip file.
         self.ubl_certificate_secret_arn: str =\
@@ -59,6 +59,13 @@ def __init__(self):
         # If false, then we use Amazon DocumentDB to back the render farm.
         self.deploy_mongo_db: bool = False
 
+        # Whether to enable Deadline Secrets Management.
+        self.enable_secrets_management: bool = True
+
+        # A Secret in AWS SecretsManager that stores the admin credentials for Deadline Secrets Management.
+        # If not defined and Secrets Management is enabled, an AWS Secret with admin credentials will be generated.
+        self.secrets_management_secret_arn: Optional[str] = None
+
         # This is only relevant if deploy_mongo_db is True.
         #
         # Change this value to MongoDbSsplLicenseAcceptance.USER_ACCEPTS_SSPL

examples/deadline/All-In-AWS-Infrastructure-Basic/python/package/lib/compute_tier.py

@@ -1,6 +1,7 @@
 # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 # SPDX-License-Identifier: Apache-2.0
 
+import os
 from dataclasses import dataclass
 from typing import (
     List,
@@ -16,7 +17,8 @@
     BastionHostLinux,
     IMachineImage,
     IVpc,
-    Port
+    Port,
+    SubnetSelection
 )
 from aws_cdk.aws_s3_assets import (
   Asset
@@ -34,7 +36,8 @@
     WorkerInstanceFleet,
 )
 
-import os
+
+from . import subnets
 
 @dataclass
 class ComputeTierProps(StackProps):
@@ -101,6 +104,9 @@ def __init__(self, scope: Construct, stack_id: str, *, props: ComputeTierProps,
             self,
             'HealthMonitor',
             vpc=props.vpc,
+            vpc_subnets=SubnetSelection(
+                subnet_group_name=subnets.INFRASTRUCTURE.name
+            ),
             # TODO - Evaluate deletion protection for your own needs. This is set to false to
             # cleanly remove everything when this stack is destroyed. If you would like to ensure
             # that this resource is not accidentally deleted, you should set this to true.
@@ -111,6 +117,9 @@ def __init__(self, scope: Construct, stack_id: str, *, props: ComputeTierProps,
             self,
             'WorkerFleet',
             vpc=props.vpc,
+            vpc_subnets=SubnetSelection(
+                subnet_group_name=subnets.WORKERS.name
+            ),
             render_queue=props.render_queue,
             worker_machine_image=props.worker_machine_image,
             health_monitor=self.health_monitor,

examples/deadline/All-In-AWS-Infrastructure-Basic/python/package/lib/network_tier.py

@@ -12,7 +12,6 @@
     GatewayVpcEndpointAwsService,
     InterfaceVpcEndpointAwsService,
     Vpc,
-    SubnetConfiguration,
     SubnetSelection,
     SubnetType
 )
@@ -21,6 +20,9 @@
     PrivateHostedZone
 )
 
+from . import subnets
+
+
 _INTERFACE_ENDPOINT_SERVICES = [
     {'name': 'CLOUDWATCH', 'service': InterfaceVpcEndpointAwsService.CLOUDWATCH},
     {'name': 'CLOUDWATCH_EVENTS', 'service': InterfaceVpcEndpointAwsService.CLOUDWATCH_EVENTS},
@@ -61,16 +63,25 @@ def __init__(self, scope: Construct, stack_id: str, **kwargs) -> None:
             'Vpc',
             max_azs=2,
             subnet_configuration=[
-                SubnetConfiguration(
-                    name='Public',
-                    subnet_type=SubnetType.PUBLIC,
-                    cidr_mask=28
-                ),
-                SubnetConfiguration(
-                    name='Private',
-                    subnet_type=SubnetType.PRIVATE,
-                    cidr_mask=18  # 16,382 IP addresses
-                )
+                # Subnets for undistinguished render farm back-end infrastructure
+                subnets.INFRASTRUCTURE,
+                # Subnets for publicly accessible infrastructure
+                subnets.PUBLIC,
+                # Subnets for the Render Queue Application Load Balancer (ALB).
+                #
+                # It is considered good practice to put a load blanacer in dedicated subnets. Additionally, the subnets
+                # must have a CIDR block with a bitmask of at least /27 and at least 8 free IP addresses per subnet.
+                # ALBs can scale up to a maximum of 100 IP addresses distributed across all subnets. Assuming only 2 AZs
+                # (the minimum) we should have 50 IPs per subnet = CIDR mask of /26
+                #
+                # See:
+                # - https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html#subnets-load-balancer
+                # - https://github.com/aws/aws-rfdk/blob/release/packages/aws-rfdk/lib/deadline/README.md#render-queue-subnet-placement
+                subnets.RENDER_QUEUE_ALB,
+                # Subnets for Usage-Based Licensing
+                subnets.USAGE_BASED_LICENSING,
+                # Subnets for the Worker instances
+                subnets.WORKERS
             ]
         )
         # VPC flow logs are a security best-practice as they allow us

examples/deadline/All-In-AWS-Infrastructure-Basic/python/package/lib/service_tier.py

@@ -15,8 +15,7 @@
     BlockDevice,
     BlockDeviceVolume,
     IVpc,
-    SubnetSelection,
-    SubnetType
+    SubnetSelection
 )
 from aws_cdk.aws_elasticloadbalancingv2 import (
     ApplicationProtocol
@@ -42,12 +41,15 @@
     RenderQueueTrafficEncryptionProps,
     RenderQueueExternalTLSProps,
     Repository,
+    SecretsManagementProps,
     ThinkboxDockerImages,
     UsageBasedLicense,
     UsageBasedLicensing,
     VersionQuery,
 )
 
+from . import subnets
+
 
 @dataclass
 class ServiceTierProps(StackProps):
@@ -72,6 +74,10 @@ class ServiceTierProps(StackProps):
     deadline_version: str
     # Whether the AWS Thinkbox End-User License Agreement is accepted or not
     accept_aws_thinkbox_eula: AwsThinkboxEulaAcceptance
+    # Whether to enable Deadline Secrets Management.
+    enable_secrets_management: bool
+    # The ARN of the AWS Secret containing the admin credentials for Deadline Secrets Management.
+    secrets_management_secret_arn: typing.Optional[str]
 
 
 class ServiceTier(Stack):
@@ -99,7 +105,7 @@ def __init__(self, scope: Construct, stack_id: str, *, props: ServiceTierProps,
             'Bastion',
             vpc=props.vpc,
             subnet_selection=SubnetSelection(
-                subnet_type=SubnetType.PUBLIC
+                subnet_group_name=subnets.PUBLIC.name
             ),
             block_devices=[
                 BlockDevice(
@@ -122,15 +128,25 @@ def __init__(self, scope: Construct, stack_id: str, *, props: ServiceTierProps,
             version=props.deadline_version
         )
 
+        secrets_management_settings = SecretsManagementProps(
+            enabled = props.enable_secrets_management
+        )
+        if props.enable_secrets_management and props.secrets_management_secret_arn is not None:
+            secrets_management_settings["credentials"] = Secret.from_secret_arn(self, 'SMAdminUser', props.secrets_management_secret_arn)
+
         repository = Repository(
             self,
             'Repository',
             vpc=props.vpc,
+            vpc_subnets=SubnetSelection(
+                subnet_group_name=subnets.INFRASTRUCTURE.name
+            ),
             database=props.database,
             file_system=props.mountable_file_system,
             repository_installation_timeout=Duration.minutes(20),
             repository_installation_prefix='/',
-            version=self.version
+            version=self.version,
+            secrets_management_settings=secrets_management_settings
         )
 
         images = ThinkboxDockerImages(
@@ -155,6 +171,22 @@ def __init__(self, scope: Construct, stack_id: str, *, props: ServiceTierProps,
             self,
             'RenderQueue',
             vpc=props.vpc,
+            vpc_subnets=SubnetSelection(
+                subnet_group_name=subnets.INFRASTRUCTURE.name
+            ),
+            # It is considered good practice to put the Render Queue's load blanacer in dedicated subnets because:
+            #
+            # 1. Deadline Secrets Management identity registration settings will be scoped down to least-privilege
+            #
+            #    (see https://github.com/aws/aws-rfdk/blob/release/packages/aws-rfdk/lib/deadline/README.md#render-queue-subnet-placement)
+            #
+            # 2. The load balancer can scale to use IP addresses in the subnet without conflicts from other AWS
+            #    resources
+            #
+            #    (see https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html#subnets-load-balancer)
+            vpc_subnets_alb=SubnetSelection(
+                subnet_group_name=subnets.RENDER_QUEUE_ALB.name
+            ),
             images=images,
             repository=repository,
             hostname=RenderQueueHostNameProps(
@@ -195,6 +227,9 @@ def __init__(self, scope: Construct, stack_id: str, *, props: ServiceTierProps,
                 self,
                 'UsageBasedLicensing',
                 vpc=props.vpc,
+                vpc_subnets=SubnetSelection(
+                    subnet_group_name=subnets.USAGE_BASED_LICENSING.name
+                ),
                 images=images,
                 licenses=props.ubl_licenses,
                 render_queue=self.render_queue,