docs: add construct architecture diagrams (#508)

aws · Aug 3, 2021 · b83c0e0 · b83c0e0
1 parent bf5bbb9
commit b83c0e0
Show file tree

Hide file tree

Showing 24 changed files with 255 additions and 8 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -75,10 +75,10 @@ The developers actively use Linux for development, but macOS and the Windows Sub
 
 To build, we use the [jsii/superchain docker container](https://hub.docker.com/r/jsii/superchain).
 
-1. Acquire the latest `jsii/superchain` docker image, if you do not already have it.
+1. Acquire the latest `jsii/superchain:node14` docker image, if you do not already have it.
 
     ```bash
-    docker pull jsii/superchain
+    docker pull jsii/superchain:node14
     ```
 
 2. Enter the docker container
@@ -180,6 +180,15 @@ git clean -fdx
 - [ ] Docs
   - __jsdocs__: All public APIs documented
   - __README__: README and/or documentation topic updated
+  - __Architecture Diagrams__: Any new constructs with non-trivial complexity should have accompanying architectural
+    diagrams. Similarly, any changes to a construct's architecture should also be reflected in the architecture
+    diagrams.
+
+    Please read the [architecture diagrams README for more details](packages/aws-rfdk/docs/diagrams/README.md).
+
+  - __Breaking?__: add upgrade documentation to `packages/aws-rfdk/docs/upgrade` which clearly guides users through the
+    process of assessing whether they are impacted by the breaking change and to determine what action they should take
+    to upgrade
 - [ ] Title and Description
   - __Change type__: title prefixed with **fix**, **feat** and module name in parens, which will appear in changelog
   - __Title__: use lower-case and doesn't end with a period

diff --git a/packages/aws-rfdk/docs/diagrams/README.md b/packages/aws-rfdk/docs/diagrams/README.md
@@ -0,0 +1,99 @@
+RFDK Architecture Diagrams
+==========================
+
+RFDK uses https://app.diagrams.net for our diagrams. We use the `.svg` file format and embed the draw.io diagram inside.
+
+## Directory Structure
+The following directory structure convention is used to place the architecture diagrams:
+
+    packages/aws-rfdk/docs/diagrams/<package>/<ConstructName>.svg
+
+Or visually speaking:
+
+```
+aws-rfdk/
+└── packages
+    └── aws-rfdk
+        └── docs
+            └── diagrams
+                ├── core
+                │   ├── <ConstructName>.svg
+                │   └── ...
+                ├── deadline
+                │   ├── <ConstructName>.svg
+                │   └── ...
+                └── <additional_package>
+                    ├── <ConstructName>.svg
+                    └── ...
+```
+
+## Referencing the Architecture Diagrams in Code
+
+Architecture diagrams should be embedded in construct JSDoc strings. They can be expressed using Markdown image syntax:
+
+```md
+![architecture diagram](/diagrams/<package>/<ConstructName>.svg)
+```
+
+where the URL is an absolute POSIX path rooted from `aws-rfdk/packages/aws-rfdk/docs`. The RFDK API reference docs
+only process image URLs that begin with `/diagrams`. Putting this together, the JSDoc header for construct `MyConstruct`
+in the `core` package would look:
+
+```ts
+import { Construct } from '@aws-cdk/core';
+
+/**
+ * Documentation for MyConstruct
+ *
+ * ![architecture diagram](/diagrams/core/MyConstruct.svg)
+ */
+export class MyConstruct extends Construct {
+  // ...
+}
+```
+
+## Diagram Style Conventions
+
+Please make a best-effort to match the design language of the architecture diagrams. Some details include:
+
+### General
+
+*   10pt Helvetica font
+*   Orthogonal (horizontal and vertical only) routing of lines with no curves
+*   For connecting lines that overlap other lines of the same color, use the **Arc** for the **Line jumps** line
+    property 
+
+### Constructs
+
+
+#### Primary Construct Overlay
+
+For per-construct architecture diagrams, the resources deployed by the primary construct being documented should have an
+overlay rectangle with the following properties:
+
+| Property              | Value                         |
+| --------------------- | ----------------------------- |
+| **Fill color**        | 0E7017                        |
+| **Fill opacity**      | 8                             |
+| **Font color**        | 0A5211                        |
+| **Font**              | Helvetica, 16pt               |
+| **Text Alignment**    | Top Center / Bottom Center    |
+
+It should be brought to the top of the layers. See the `SpotEventPluginFleet` diagram for an example:
+
+![SpotEventPluginFleet architecture diagram](./deadline/SpotEventPluginFleet.svg)
+
+### Interfaces
+
+In a construct architecture diagram, illustrate the interfaces between other RFDK constructs. For these related
+constructs, only resources that participate in the interface need to be included in the diagram. For example, see the
+following architecture diagram for the `RenderQueue`:
+
+![RenderQueue architecture diagram](./deadline/RenderQueue.svg)
+
+### Resources and Services
+
+When a CloudFormation resource is present in a diagram, its corresponding service should have its own icon in the
+diagram with a connnecting line between the service and the resource. These services and resources are color-coded
+as provided by the **AWS19** draw.io icon library. The color of the font and line connecting the service to the resource
+should match the service/resource fill color to help visually connect and group the resources and services.
diff --git a/packages/aws-rfdk/docs/diagrams/core/HealthMonitor.svg b/packages/aws-rfdk/docs/diagrams/core/HealthMonitor.svg
diff --git a/packages/aws-rfdk/docs/diagrams/core/ImportedAcmCertificate.svg b/packages/aws-rfdk/docs/diagrams/core/ImportedAcmCertificate.svg
diff --git a/packages/aws-rfdk/docs/diagrams/core/X509CertificatePem.svg b/packages/aws-rfdk/docs/diagrams/core/X509CertificatePem.svg
diff --git a/packages/aws-rfdk/docs/diagrams/core/X509CertificatePkcs12.svg b/packages/aws-rfdk/docs/diagrams/core/X509CertificatePkcs12.svg
diff --git a/packages/aws-rfdk/docs/diagrams/deadline/ConfigureSpotEventPlugin.svg b/packages/aws-rfdk/docs/diagrams/deadline/ConfigureSpotEventPlugin.svg
diff --git a/packages/aws-rfdk/docs/diagrams/deadline/RenderQueue.svg b/packages/aws-rfdk/docs/diagrams/deadline/RenderQueue.svg
diff --git a/packages/aws-rfdk/docs/diagrams/deadline/Repository.svg b/packages/aws-rfdk/docs/diagrams/deadline/Repository.svg
diff --git a/packages/aws-rfdk/docs/diagrams/deadline/SpotEventPluginFleet.svg b/packages/aws-rfdk/docs/diagrams/deadline/SpotEventPluginFleet.svg
diff --git a/packages/aws-rfdk/docs/diagrams/deadline/UsageBasedLicensing.svg b/packages/aws-rfdk/docs/diagrams/deadline/UsageBasedLicensing.svg
diff --git a/packages/aws-rfdk/docs/diagrams/deadline/WorkerInstanceFleet.svg b/packages/aws-rfdk/docs/diagrams/deadline/WorkerInstanceFleet.svg
diff --git a/packages/aws-rfdk/docs/diagrams/tests/svg-embedded-drawio.test.ts b/packages/aws-rfdk/docs/diagrams/tests/svg-embedded-drawio.test.ts
@@ -0,0 +1,75 @@
+/**
+ * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+import * as fs from 'fs';
+import { promises as fsPromises } from 'fs';
+import * as path from 'path';
+
+/**
+ * This is the regular expression that validates that the draw.io diagram is embedded within the SVG file.
+ *
+ * draw.io embeds the diagram as a "content" XML attribute of the <svg> element. This looks like:
+ *
+ * <svg ... content="...embedded SVG diagram...">
+ *   ...
+ * </svg>
+ *
+ * When you choose to not include a copy of the draw.io diagram, this attribute is not written to the file.
+ *
+ * This is a very simple regular expression that will match the opening <svg> tag and ensure it has a non-empty
+ * XML attribute named "content". While this validation doesn't assert the validity of the content attribute,
+ * it will catch the common mistake of forgetting to include an embedded copy of the draw.io diagram.
+ */
+const DRAW_IO_EMBEDDED_XML_ATTR_REGEX = /<svg( [^>]+)?content="[^"]+?"[^>]*?>/;
+const DIAGRAM_ROOT = path.join(__dirname, '..');
+
+/**
+ * Returns all `.svg` files that are descendants of the `packages/aws-rfdk/docs/diagrams` directory.
+ */
+function findSvgDiagrams(dirPath?: string): string[] {
+  if (!dirPath) {
+    dirPath = DIAGRAM_ROOT;
+  }
+
+  let diagrams: string[] = [];
+
+  const listings = fs.readdirSync(dirPath);
+
+  for (const listing of listings) {
+    const listingPath = path.join(dirPath, listing);
+    const stat = fs.statSync(listingPath);
+    if (stat.isDirectory()) {
+      const dirDiagrams = findSvgDiagrams(listingPath);
+      diagrams = diagrams.concat(...dirDiagrams);
+    }
+    else if (path.extname(listing).toLowerCase() === '.svg') {
+      diagrams.push(listingPath);
+    }
+  }
+
+  return diagrams;
+}
+
+describe('diagrams', () => {
+  const diagrams = findSvgDiagrams();
+
+  describe('have draw.io diagrams embedded', () => {
+
+    diagrams.forEach(diagram => {
+      const relativeDiagramPath = path.relative(DIAGRAM_ROOT, diagram);
+
+      // We use string interpolation below, otherwise eslint incorrectly believes that the test name is not a valid
+      // string
+      test(`${relativeDiagramPath}`, async () => {
+        // GIVEN
+        const diagramContents = await fsPromises.readFile(diagram, { encoding: 'utf8' });
+
+        // THEN
+        expect(diagramContents).toMatch(DRAW_IO_EMBEDDED_XML_ATTR_REGEX);
+      });
+    });
+
+  });
+});
diff --git a/packages/aws-rfdk/lib/core/README.md b/packages/aws-rfdk/lib/core/README.md
@@ -135,6 +135,8 @@ const exportingLogGroup = new ExportingLogGroup(stack, 'ExportingLogGroup', {
 
 ## HealthMonitor
 
+![architecture diagram](../../docs/diagrams/core/HealthMonitor.svg)
+
 In order to monitor a heartbeat reported by each render node and to automatically reboot instances that are failing, which will help to avoid extra costs, you can use `HealthMonitor` instance:
 
 ```ts
@@ -658,6 +660,8 @@ RFDK provides the following constructs for working with X509 certificates: `X509
 
 ### X509CertificatePem
 
+![architecture diagram](../../docs/diagrams/core/X509CertificatePem.svg)
+
 `X509CertificatePem` provides a mechanism for generating X.509 certificates. This construct will create the following resources as secrets in Secret Manager:
 1. An X509 certificate in PEM format
 2. A private key in PEM format
@@ -731,6 +735,8 @@ cert.grantFullRead(instance.grantPrincipal);
 
 ### X509CertificatePkcs12
 
+![architecture diagram](../../docs/diagrams/core/X509CertificatePkcs12.svg)
+
 In order to generate a PKCS #12 file from an X.509 certificate in PEM format you can use `X509CertificatePkcs12` construct:
 
 ```ts
@@ -754,6 +760,8 @@ const pkcs12Cert = new X509CertificatePkcs12(stack, 'CertPkcs12', {
 
 ### ImportedAcmCertificate
 
+![architecture diagram](../../docs/diagrams/core/ImportedAcmCertificate.svg)
+
 You might need to import your X.509 certificate stored as a Secret into the AWS Certificate Manager. In this case, you can use `ImportedAcmCertificate` to do that:
 
 ```ts

diff --git a/packages/aws-rfdk/lib/core/lib/health-monitor.ts b/packages/aws-rfdk/lib/core/lib/health-monitor.ts
@@ -278,6 +278,8 @@ abstract class HealthMonitorBase extends Construct implements IHealthMonitor {
  * being sufficiently unhealthy to warrant termination.
  * This lambda is triggered by CloudWatch alarms via SNS (Simple Notification Service).
  *
+ * ![architecture diagram](/diagrams/core/HealthMonitor.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - Application Load Balancer(s) doing frequent pings to the workers.

diff --git a/packages/aws-rfdk/lib/core/lib/imported-acm-certificate.ts b/packages/aws-rfdk/lib/core/lib/imported-acm-certificate.ts
@@ -74,6 +74,8 @@ export interface ImportedAcmCertificateProps {
  * AWS Certificate Manager (ACM). It uses an AWS Lambda Function to extract the certificate from Secrets in AWS SecretsManager
  * and then import it into ACM. The interface is intended to be used with the {@link X509CertificatePem} Construct.
  *
+ * ![architecture diagram](/diagrams/core/ImportedAcmCertificate.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - DynamoDB Table - Used for tracking resources created by the Custom Resource.

diff --git a/packages/aws-rfdk/lib/core/lib/x509-certificate.ts b/packages/aws-rfdk/lib/core/lib/x509-certificate.ts
@@ -243,6 +243,8 @@ abstract class X509CertificateBase extends Construct {
  * The cost of four AWS SecretsManager Secrets in the deployed region.
  * The other resources created by this construct have negligible ongoing costs.
  *
+ * ![architecture diagram](/diagrams/core/X509CertificatePem.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - DynamoDB Table - Used for tracking resources created by the Custom Resource.
@@ -400,6 +402,8 @@ export interface IX509CertificatePkcs12 extends IConstruct {
  * the result is stored in a Secret. The PKCS #12 file is password protected with a passphrase that is randomly
  * generated and stored in a Secret.
  *
+ * ![architecture diagram](/diagrams/core/X509CertificatePkcs12.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - DynamoDB Table - Used for tracking resources created by the CustomResource.

diff --git a/packages/aws-rfdk/lib/deadline/README.md b/packages/aws-rfdk/lib/deadline/README.md
@@ -36,6 +36,8 @@ _**Note:** RFDK constructs currently support Deadline 10.1.9 and later, unless o
 
 ## Configure Spot Event Plugin
 
+![architecture diagram](../../docs/diagrams/deadline/ConfigureSpotEventPlugin.svg)
+
 The [Spot Event Plugin](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html) can scale cloud-based EC2 Spot instances dynamically based on the queued Jobs and Tasks in the Deadline Database. It associates a Spot Fleet Request with named Deadline Worker Groups, allowing multiple Spot Fleets with different hardware and software specifications to be launched for different types of Jobs based on their Group assignment.
 
 The `ConfigureSpotEventPlugin` construct has two main responsibilities:
@@ -98,19 +100,15 @@ const spotEventPluginConfig = new ConfigureSpotEventPlugin(this, 'ConfigureSpotE
 
 ## Render Queue
 
+![architecture diagram](../../docs/diagrams/deadline/RenderQueue.svg)
+
 The `RenderQueue` is the central service of a Deadline render farm. It consists of the following components:
 
 - **Deadline Repository** - The repository that initializes the persistent data schema used by Deadline such as job information, connected workers, rendered output files, etc.
 - **Deadline Remote Connection Server (RCS)** - The central server that all Deadline applications connect to. The RCS contains the core business logic to manage the render farm.
 
 The `RenderQueue` construct sets up the RCS and configures it to communicate with the Repository and to listen for requests using the configured protocol (HTTP or HTTPS). Docker container images are used to deploy the `RenderQueue` as a fleet of instances within an Elastic Container Service (ECS) cluster. This fleet of instances is load balanced by an [Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html) which has built-in health monitoring functionality that can be configured through this construct.
 
----
-
-_**Note:** The number of instances running the Render Queue is currently limited to a maximum of one._
-
----
-
 The following example outlines how to construct a `RenderQueue`:
 
 ```ts
@@ -167,6 +165,8 @@ const renderQueue = new RenderQueue(stack, 'RenderQueue', {
 
 ## Repository
 
+![architecture diagram](../../docs/diagrams/deadline/Repository.svg)
+
 The `Repository` contains the central database and file system used by Deadline. An EC2 instance is temporarily created to run the Deadline Repository installer which configures the database and file system. This construct has optional parameters for the database and file system to use, giving you the option to either provide your own resources or to have the construct create its own. Log messages emitted by the construct are forwarded to CloudWatch via a CloudWatch agent.
 
 You can create a `Repository` like this:
@@ -226,6 +226,8 @@ repository.configureClientInstance({
 
 ## Spot Event Plugin Fleet
 
+![architecture diagram](../../docs/diagrams/deadline/SpotEventPluginFleet.svg)
+
 This construct represents a Spot Fleet launched by the [Deadline's Spot Event Plugin](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html) from the [Spot Fleet Request Configuration](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/event-spot.html#spot-fleet-request-configurations). The construct itself doesn't create a Spot Fleet Request, but creates all the required resources to be used in the Spot Fleet Request Configuration.
 
 This construct is expected to be used as an input to the [ConfigureSpotEventPlugin](#configure-spot-event-plugin) construct. `ConfigureSpotEventPlugin` construct will generate a Spot Fleet Request Configuration from each provided `SpotEventPluginFleet` and will set these configurations to the Spot Event Plugin.
@@ -440,6 +442,8 @@ const ubl = new UsageBasedLicensing(scope, 'RenderQueue', {
 
 ## Usage-Based Licensing (UBL)
 
+![architecture diagram](../../docs/diagrams/deadline/UsageBasedLicensing.svg)
+
 Usage-Based Licensing is an on-demand licensing model (see [Deadline Documentation](https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/licensing-usage-based.html)). The RFDK supports this type of licensing with the `UsageBasedLicensing` construct. This construct contains the following components:
 
 - **Deadline License Forwarder** - Forwards licenses to Deadline Workers that are rendering jobs.
@@ -504,6 +508,8 @@ const version = VersionQuery.exact(stack, 'ExactVersion', {
 
 ## Worker Fleet
 
+![architecture diagram](../../docs/diagrams/deadline/WorkerInstanceFleet.svg)
+
 A `WorkerInstanceFleet` represents a fleet of instances that are the render nodes of your render farm. These instances are created in an [`AutoScalingGroup`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-autoscaling.AutoScalingGroup.html) with a provided AMI that should have Deadline Client installed as well as any desired render applications. Each of the instances will configure itself to connect to the specified [`RenderQueue`](#renderqueue) so that they are able to communicate with Deadline to pick up render jobs. Any logs emitted by the workers are sent to CloudWatch via a CloudWatch agent.
 
 You can create a `WorkerInstanceFleet` like this:

diff --git a/packages/aws-rfdk/lib/deadline/lib/configure-spot-event-plugin.ts b/packages/aws-rfdk/lib/deadline/lib/configure-spot-event-plugin.ts
@@ -335,6 +335,8 @@ export interface ConfigureSpotEventPluginProps {
  *  - A policy to pass a fleet and instance role
  *  - A policy to create tags for spot fleet requests
  *
+ * ![architecture diagram](/diagrams/deadline/ConfigureSpotEventPlugin.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - An AWS Lambda that is used to connect to the render queue, and save Spot Event Plugin configurations.

diff --git a/packages/aws-rfdk/lib/deadline/lib/render-queue.ts b/packages/aws-rfdk/lib/deadline/lib/render-queue.ts
@@ -172,6 +172,8 @@ abstract class RenderQueueBase extends Construct implements IRenderQueue {
  * Most Deadline clients will connect to a Deadline render farm via the the RenderQueue. The API provides Deadline
  * clients access to Deadline's database and repository file-system in a way that is secure, performant, and scalable.
  *
+ * ![architecture diagram](/diagrams/deadline/RenderQueue.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - An Amazon Elastic Container Service (ECS) cluster.

diff --git a/packages/aws-rfdk/lib/deadline/lib/repository.ts b/packages/aws-rfdk/lib/deadline/lib/repository.ts
@@ -402,6 +402,8 @@ export interface RepositoryProps {
  * and the deployment will continue, otherwise the the deployment will be cancelled.
  * In either case the instance will be cleaned up.
  *
+ * ![architecture diagram](/diagrams/deadline/Repository.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - Encrypted Amazon Elastic File System (EFS) - If no file system is provided.

diff --git a/packages/aws-rfdk/lib/deadline/lib/spot-event-plugin-fleet.ts b/packages/aws-rfdk/lib/deadline/lib/spot-event-plugin-fleet.ts
@@ -261,6 +261,8 @@ export interface ISpotEventPluginFleet extends IConnectable, IScriptHost, IGrant
  * required for the Spot Fleet Request and generates the Spot Fleet Configuration setting:
  * a one to one mapping between a Deadline Group and Spot Fleet Request Configurations.
  *
+ * ![architecture diagram](/diagrams/deadline/SpotEventPluginFleet.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - An Instance Role, corresponding IAM Policy and an Instance Profile.

diff --git a/packages/aws-rfdk/lib/deadline/lib/usage-based-licensing.ts b/packages/aws-rfdk/lib/deadline/lib/usage-based-licensing.ts
@@ -445,6 +445,8 @@ export interface UsageBasedLicensingProps {
  * Note: This construct does not currently implement the Deadline License Forwarder's Web Forwarding functionality.
  * This construct is not usable in any China region.
  *
+ * ![architecture diagram](/diagrams/deadline/UsageBasedLicensing.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - The Auto Scaling Group (ASG) added to the Amazon Elastic Container Service cluster that is hosting the Deadline

diff --git a/packages/aws-rfdk/lib/deadline/lib/worker-fleet.ts b/packages/aws-rfdk/lib/deadline/lib/worker-fleet.ts
@@ -313,6 +313,8 @@ abstract class WorkerInstanceFleetBase extends Construct implements IWorkerFleet
  * When the worker fleet is deployed if it has been provided a HealthMonitor the Worker fleet will register itself against the Monitor
  * to ensure that the fleet remains healthy.
  *
+ * ![architecture diagram](/diagrams/deadline/WorkerInstanceFleet.svg)
+ *
  * Resources Deployed
  * ------------------------
  * - An EC2 Auto Scaling Group to maintain the number of instances.