feat(pipelines): allow disabling use of change sets (#21619)

Possibility to change deployment way for whole pipeline, for single stage or for single stack in stage.

Fixes #20827

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

packages/@aws-cdk/pipelines/README.md

@@ -544,6 +544,32 @@ const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
 });
 ```
 
+#### Deploying without change sets
+
+Deployment is done by default with `CodePipeline` engine using change sets,
+i.e. to first create a change set and then execute it. This allows you to inject
+steps that inspect the change set and approve or reject it, but failed deployments
+are not retryable and creation of the change set costs time.
+
+The creation of change sets can be switched off by setting `useChangeSets: false`:
+
+```ts
+declare const synth: pipelines.ShellStep;
+
+class PipelineStack extends Stack {
+  constructor(scope: Construct, id: string, props?: StackProps) {
+    super(scope, id, props);
+
+    const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
+      synth,
+
+      // Disable change set creation and make deployments in pipeline as single step
+      useChangeSets: false,
+    });
+  }
+}
+```
+
 ### Validation
 
 Every `addStage()` and `addWave()` command takes additional options. As part of these options,
@@ -867,11 +893,11 @@ because those values are passed in directly to the underlying `codepipeline.Pipe
 
 Docker can be used in 3 different places in the pipeline:
 
-* If you are using Docker image assets in your application stages: Docker will
+- If you are using Docker image assets in your application stages: Docker will
   run in the asset publishing projects.
-* If you are using Docker image assets in your stack (for example as
+- If you are using Docker image assets in your stack (for example as
   images for your CodeBuild projects): Docker will run in the self-mutate project.
-* If you are using Docker to bundle file assets anywhere in your project (for
+- If you are using Docker to bundle file assets anywhere in your project (for
   example, if you are using such construct libraries as
   `@aws-cdk/aws-lambda-nodejs`): Docker will run in the
   *synth* project.
@@ -1069,26 +1095,26 @@ $ npx cdk bootstrap \
 
 These command lines explained:
 
-* `npx`: means to use the CDK CLI from the current NPM install. If you are using
+- `npx`: means to use the CDK CLI from the current NPM install. If you are using
   a global install of the CDK CLI, leave this out.
-* `--profile`: should indicate a profile with administrator privileges that has
+- `--profile`: should indicate a profile with administrator privileges that has
   permissions to provision a pipeline in the indicated account. You can leave this
   flag out if either the AWS default credentials or the `AWS_*` environment
   variables confer these permissions.
-* `--cloudformation-execution-policies`: ARN of the managed policy that future CDK
+- `--cloudformation-execution-policies`: ARN of the managed policy that future CDK
   deployments should execute with. By default this is `AdministratorAccess`, but
   if you also specify the `--trust` flag to give another Account permissions to
   deploy into the current account, you must specify a value here.
-* `--trust`: indicates which other account(s) should have permissions to deploy
+- `--trust`: indicates which other account(s) should have permissions to deploy
   CDK applications into this account. In this case we indicate the Pipeline's account,
   but you could also use this for developer accounts (don't do that for production
   application accounts though!).
-* `--trust-for-lookup`: gives a more limited set of permissions to the
+- `--trust-for-lookup`: gives a more limited set of permissions to the
   trusted account, only allowing it to look up values such as availability zones, EC2 images and
   VPCs. `--trust-for-lookup` does not give permissions to modify anything in the account.
   Note that `--trust` implies `--trust-for-lookup`, so you don't need to specify
   the same acocunt twice.
-* `aws://222222222222/us-east-2`: the account and region we're bootstrapping.
+- `aws://222222222222/us-east-2`: the account and region we're bootstrapping.
 
 > Be aware that anyone who has access to the trusted Accounts **effectively has all
 > permissions conferred by the configured CloudFormation execution policies**,
@@ -1126,10 +1152,10 @@ The "new" bootstrap stack (obtained by running `cdk bootstrap` with
 `CDK_NEW_BOOTSTRAP=1`) is slightly more elaborate than the "old" stack. It
 contains:
 
-* An S3 bucket and ECR repository with predictable names, so that we can reference
+- An S3 bucket and ECR repository with predictable names, so that we can reference
   assets in these storage locations *without* the use of CloudFormation template
   parameters.
-* A set of roles with permissions to access these asset locations and to execute
+- A set of roles with permissions to access these asset locations and to execute
   CloudFormation, assumable from whatever accounts you specify under `--trust`.
 
 It is possible and safe to migrate from the old bootstrap stack to the new
@@ -1209,15 +1235,15 @@ very nature the library cannot take care of everything.
 
 We therefore expect you to mind the following:
 
-* Maintain dependency hygiene and vet 3rd-party software you use. Any software you
+- Maintain dependency hygiene and vet 3rd-party software you use. Any software you
   run on your build machine has the ability to change the infrastructure that gets
   deployed. Be careful with the software you depend on.
 
-* Use dependency locking to prevent accidental upgrades! The default `CdkSynths` that
+- Use dependency locking to prevent accidental upgrades! The default `CdkSynths` that
   come with CDK Pipelines will expect `package-lock.json` and `yarn.lock` to
   ensure your dependencies are the ones you expect.
 
-* Credentials to production environments should be short-lived. After
+- Credentials to production environments should be short-lived. After
   bootstrapping and the initial pipeline provisioning, there is no more need for
   developers to have access to any of the account credentials; all further
   changes can be deployed through git. Avoid the chances of credentials leaking
@@ -1292,7 +1318,7 @@ use CDK Pipelines to build pipelines backed by other deployment engines.
 Here is a list of CDK Libraries that integrate CDK Pipelines with
 alternative deployment engines:
 
-* GitHub Workflows: [`cdk-pipelines-github`](https://github.com/cdklabs/cdk-pipelines-github)
+- GitHub Workflows: [`cdk-pipelines-github`](https://github.com/cdklabs/cdk-pipelines-github)
 
 ## Troubleshooting
 
@@ -1329,11 +1355,11 @@ If you see this error during the **Synth** step, it means that CodeBuild
 is expecting to find a `cdk.out` directory in the root of your CodeBuild project,
 but the directory wasn't there. There are two common causes for this:
 
-* `cdk synth` is not being executed: `cdk synth` used to be run
+- `cdk synth` is not being executed: `cdk synth` used to be run
   implicitly for you, but you now have to explicitly include the command.
   For NPM-based projects, add `npx cdk synth` to the end of the `commands`
   property, for other languages add `npm install -g aws-cdk` and `cdk synth`.
-* Your CDK project lives in a subdirectory: you added a `cd <somedirectory>` command
+- Your CDK project lives in a subdirectory: you added a `cd <somedirectory>` command
   to the list of commands; don't forget to tell the `ScriptStep` about the
   different location of `cdk.out`, by passing `primaryOutputDirectory: '<somedirectory>/cdk.out'`.
 
@@ -1426,9 +1452,9 @@ all, and commit a file called `cdk.context.json` with the right lookup values in
 
 If you do want to do lookups in the pipeline, the cause is one of the following:
 
-* The target environment has not been bootstrapped; OR
-* The target environment has been bootstrapped without the right `--trust` relationship; OR
-* The CodeBuild execution role does not have permissions to call `sts:AssumeRole`.
+- The target environment has not been bootstrapped; OR
+- The target environment has been bootstrapped without the right `--trust` relationship; OR
+- The CodeBuild execution role does not have permissions to call `sts:AssumeRole`.
 
 See the section called **Context Lookups** for more information on using this feature.
 
@@ -1452,8 +1478,8 @@ following:
 
 An "S3 Access Denied" error can have two causes:
 
-* Asset hashes have changed, but self-mutation has been disabled in the pipeline.
-* You have deleted and recreated the bootstrap stack, or changed its qualifier.
+- Asset hashes have changed, but self-mutation has been disabled in the pipeline.
+- You have deleted and recreated the bootstrap stack, or changed its qualifier.
 
 #### Self-mutation step has been removed
 
@@ -1495,7 +1521,7 @@ The most automated way to solve the issue is to introduce a secondary bootstrap
 that the pipeline stack looks for, a change will be detected and the impacted policies and resources will be updated.
 A hypothetical recovery workflow would look something like this:
 
-* First, for all impacted environments, create a secondary bootstrap stack:
+- First, for all impacted environments, create a secondary bootstrap stack:
 
 ```sh
 $ env CDK_NEW_BOOTSTRAP=1 npx cdk bootstrap \
@@ -1504,7 +1530,7 @@ $ env CDK_NEW_BOOTSTRAP=1 npx cdk bootstrap \
     aws://111111111111/us-east-1
 ```
 
-* Update all impacted stacks in the pipeline to use this new qualifier.
+- Update all impacted stacks in the pipeline to use this new qualifier.
 See https://docs.aws.amazon.com/cdk/latest/guide/bootstrapping.html for more info.
 
 ```ts
@@ -1516,11 +1542,11 @@ new Stack(this, 'MyStack', {
 });
 ```
 
-* Deploy the updated stacks. This will update the stacks to use the roles created in the new bootstrap stack.
-* (Optional) Restore back to the original state:
-  * Revert the change made in step #2 above
-  * Re-deploy the pipeline to use the original qualifier.
-  * Delete the temporary bootstrap stack(s)
+- Deploy the updated stacks. This will update the stacks to use the roles created in the new bootstrap stack.
+- (Optional) Restore back to the original state:
+  - Revert the change made in step #2 above
+  - Re-deploy the pipeline to use the original qualifier.
+  - Delete the temporary bootstrap stack(s)
 
 ##### Manual Alternative
 
@@ -1655,14 +1681,14 @@ const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
 There are some usability issues that are caused by underlying technology, and
 cannot be remedied by CDK at this point. They are reproduced here for completeness.
 
-* **Console links to other accounts will not work**: the AWS CodePipeline
+- **Console links to other accounts will not work**: the AWS CodePipeline
   console will assume all links are relative to the current account. You will
   not be able to use the pipeline console to click through to a CloudFormation
   stack in a different account.
-* **If a change set failed to apply the pipeline must restarted**: if a change
+- **If a change set failed to apply the pipeline must restarted**: if a change
   set failed to apply, it cannot be retried. The pipeline must be restarted from
   the top by clicking **Release Change**.
-* **A stack that failed to create must be deleted manually**: if a stack
+- **A stack that failed to create must be deleted manually**: if a stack
   failed to create on the first attempt, you must delete it using the
   CloudFormation console before starting the pipeline again by clicking
   **Release Change**.

packages/@aws-cdk/pipelines/lib/blueprint/stage-deployment.ts

@@ -117,6 +117,12 @@ export class StageDeployment {
    */
   public readonly stackSteps: StackSteps[];
 
+  /**
+   * Determine if all stacks in stage should be deployed with prepare
+   * step or not.
+   */
+  public readonly prepareStep?: boolean;
+
   private constructor(
     /** The stacks deployed in this stage */
     public readonly stacks: StackDeployment[], props: StageDeploymentProps = {}) {

packages/@aws-cdk/pipelines/lib/codepipeline/codepipeline.ts

@@ -214,6 +214,16 @@ export interface CodePipelineProps {
    * @default - A new role is created
    */
   readonly role?: iam.IRole;
+
+  /**
+   * Deploy every stack by creating a change set and executing it
+   *
+   * When enabled, creates a "Prepare" and "Execute" action for each stack. Disable
+   * to deploy the stack in one pipeline action.
+   *
+   * @default true
+   */
+  readonly useChangeSets?: boolean;
 }
 
 /**
@@ -299,6 +309,7 @@ export class CodePipeline extends PipelineBase {
   private artifacts = new ArtifactMap();
   private _synthProject?: cb.IProject;
   private readonly selfMutation: boolean;
+  private readonly useChangeSets: boolean;
   private _myCxAsmRoot?: string;
   private readonly dockerCredentials: DockerCredential[];
   private readonly cachedFnSub = new CachedFnSub();
@@ -325,6 +336,7 @@ export class CodePipeline extends PipelineBase {
     this.dockerCredentials = props.dockerCredentials ?? [];
     this.singlePublisherPerAssetType = !(props.publishAssetsInParallel ?? true);
     this.cliVersion = props.cliVersion ?? preferredCliVersion();
+    this.useChangeSets = props.useChangeSets ?? true;
   }
 
   /**
@@ -389,6 +401,7 @@ export class CodePipeline extends PipelineBase {
     const graphFromBp = new PipelineGraph(this, {
       selfMutation: this.selfMutation,
       singlePublisherPerAssetType: this.singlePublisherPerAssetType,
+      prepareStep: this.useChangeSets,
     });
     this._cloudAssemblyFileSet = graphFromBp.cloudAssemblyFileSet;
 
@@ -519,10 +532,15 @@ export class CodePipeline extends PipelineBase {
         return this.createChangeSetAction(node.data.stack);
 
       case 'execute':
-        return this.executeChangeSetAction(node.data.stack, node.data.captureOutputs);
+        return node.data.withoutChangeSet
+          ? this.executeDeploymentAction(node.data.stack, node.data.captureOutputs)
+          : this.executeChangeSetAction(node.data.stack, node.data.captureOutputs);
 
       case 'step':
         return this.actionFromStep(node, node.data.step);
+
+      default:
+        throw new Error(`CodePipeline does not support graph nodes of type '${node.data?.type}'. You are probably using a feature this CDK Pipelines implementation does not support.`);
     }
   }
 
@@ -630,6 +648,38 @@ export class CodePipeline extends PipelineBase {
     };
   }
 
+  private executeDeploymentAction(stack: StackDeployment, captureOutputs: boolean): ICodePipelineActionFactory {
+    const templateArtifact = this.artifacts.toCodePipeline(this._cloudAssemblyFileSet!);
+    const templateConfigurationPath = this.writeTemplateConfiguration(stack);
+
+    const region = stack.region !== Stack.of(this).region ? stack.region : undefined;
+    const account = stack.account !== Stack.of(this).account ? stack.account : undefined;
+
+    const relativeTemplatePath = path.relative(this.myCxAsmRoot, stack.absoluteTemplatePath);
+
+    return {
+      produceAction: (stage, options) => {
+        stage.addAction(new cpa.CloudFormationCreateUpdateStackAction({
+          actionName: options.actionName,
+          runOrder: options.runOrder,
+          stackName: stack.stackName,
+          templatePath: templateArtifact.atPath(toPosixPath(relativeTemplatePath)),
+          adminPermissions: true,
+          role: this.roleFromPlaceholderArn(this.pipeline, region, account, stack.assumeRoleArn),
+          deploymentRole: this.roleFromPlaceholderArn(this.pipeline, region, account, stack.executionRoleArn),
+          region: region,
+          templateConfiguration: templateConfigurationPath
+            ? templateArtifact.atPath(toPosixPath(templateConfigurationPath))
+            : undefined,
+          cfnCapabilities: [CfnCapabilities.NAMED_IAM, CfnCapabilities.AUTO_EXPAND],
+          variablesNamespace: captureOutputs ? stackVariableNamespace(stack) : undefined,
+        }));
+
+        return { runOrdersConsumed: 1 };
+      },
+    };
+  }
+
   private selfMutateAction(): ICodePipelineActionFactory {
     const installSuffix = this.cliVersion ? `@${this.cliVersion}` : '';