core: 'cdk synth' always synthesizes every stack

[castleadmin]

Describe the bug

If the command cdk synth 'Dev/*' is executed. All stacks are synthesized.

The same output is created in the cdk.out directory if a concrete stack is synthesized cdk synth 'Dev/ExampleStack'.

Expected Behavior

Only the Dev environment stacks should be synthesized. Only the Dev environment lambda assets should be created.

Current Behavior

All stacks are synthesized. All lambda assets are created.

Reproduction Steps

Create a project with the following files and execute cdk synth 'Dev/*' .

cdk/app.ts

import { App, Tags } from 'aws-cdk-lib';
import { SourceMapMode } from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';
import { resolve } from 'node:path';
import 'source-map-support/register';
import { Environment } from '../shared/environment';
import { ExampleStack, ExampleStackProps } from './example-stack';

export const createApp = (): App => {
  const app = new App();
  const serviceName = 'testApp';

  new EnvironmentStacks(app, 'Dev', {
    environment: Environment.Dev,
    serviceName,
    env: {
      account:
        process.env['CDK_DEV_ACCOUNT'] ??
        process.env['CDK_DEFAULT_ACCOUNT'] ??
        'not-available',
      region:
        process.env['CDK_DEV_REGION'] ??
        process.env['CDK_DEFAULT_REGION'] ??
        'not-available',
    },
    build: {
      minify: false,
      sourceMapMode: SourceMapMode.INLINE,
      tsconfig: resolve(__dirname, '../tsconfig.app.json'),
    },
  });

  new EnvironmentStacks(app, 'Stage', {
    environment: Environment.Stage,
    serviceName,
    env: {
      account:
        process.env['CDK_STAGE_ACCOUNT'] ??
        process.env['CDK_DEFAULT_ACCOUNT'] ??
        'not-available',
      region:
        process.env['CDK_STAGE_REGION'] ??
        process.env['CDK_DEFAULT_REGION'] ??
        'not-available',
    },
    build: {
      minify: true,
      sourceMapMode: SourceMapMode.EXTERNAL,
      tsconfig: resolve(__dirname, '../tsconfig.app.json'),
    },
  });

  new EnvironmentStacks(app, 'Prod', {
    environment: Environment.Prod,
    serviceName,
    env: {
      account:
        process.env['CDK_PROD_ACCOUNT'] ??
        process.env['CDK_DEFAULT_ACCOUNT'] ??
        'not-available',
      region:
        process.env['CDK_PROD_REGION'] ??
        process.env['CDK_DEFAULT_REGION'] ??
        'not-available',
    },
    build: {
      minify: true,
      sourceMapMode: SourceMapMode.EXTERNAL,
      tsconfig: resolve(__dirname, '../tsconfig.app.json'),
    },
  });

  return app;
};

export type EnvironmentStacksProps = ExampleStackProps;

export class EnvironmentStacks extends Construct {
  constructor(scope: Construct, id: string, props: EnvironmentStacksProps) {
    const { environment, serviceName } = props;

    super(scope, id);

    const exampleStack = new ExampleStack(this, 'ExampleStack', props);

    Tags.of(exampleStack).add('Environment', environment);
    Tags.of(exampleStack).add('App', serviceName);
  }
}

createApp();

cdk/example-stack.ts

import { Duration, Stack, StackProps } from 'aws-cdk-lib';
import { LogFormat, Runtime, Tracing } from 'aws-cdk-lib/aws-lambda';
import {
  Charset,
  NodejsFunction,
  OutputFormat,
  SourceMapMode,
} from 'aws-cdk-lib/aws-lambda-nodejs';
import { RetentionDays } from 'aws-cdk-lib/aws-logs';
import { Construct } from 'constructs';
import { resolve } from 'node:path';
import { Environment } from '../shared/environment';

export interface ExampleStackProps extends StackProps {
  // Define stack properties here

  environment: Environment;
  serviceName: string;
  build: {
    minify: boolean;
    sourceMapMode: SourceMapMode;
    tsconfig: string;
  };
}

export class ExampleStack extends Stack {
  constructor(scope: Construct, id: string, props: ExampleStackProps) {
    const { environment, serviceName, build, ...stackProps } = props;
    const { minify, sourceMapMode, tsconfig } = build;

    super(scope, id, stackProps);

    new NodejsFunction(this, 'ExampleFunction', {
      bundling: {
        charset: Charset.UTF8,
        format: OutputFormat.ESM,
        minify,
        sourceMap: true,
        sourceMapMode,
        tsconfig,
        nodeModules: ['@aws-lambda-powertools/logger'],
      },
      description: 'An example Lambda function.',
      entry: resolve(__dirname, '../src/example-handler.ts'),
      environment: {
        ENVIRONMENT: environment,
        POWERTOOLS_SERVICE_NAME: serviceName,
      },
      handler: 'exampleHandler',
      logFormat: LogFormat.JSON,
      logRetention: RetentionDays.THREE_MONTHS,
      memorySize: 128,
      runtime: Runtime.NODEJS_20_X,
      timeout: Duration.seconds(29),
      tracing: Tracing.ACTIVE,
    });
  }
}

shared/environment.ts

export enum Environment {
  Dev = 'Dev',
  Stage = 'Stage',
  Prod = 'Prod',
}

src/example-handler.ts

import { Context } from 'aws-lambda';

export interface ExampleEvent {
  a: number;
  b: number;
}

export interface ExampleResult {
  sum: number;
}

export const exampleHandler = async (
  event: ExampleEvent,
  context: Context,
): Promise<ExampleResult> => {
  console.log('Example log message', JSON.stringify({ event, context }, null, 2));

  return {
    sum: event.a + event.b,
  };
};

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.110.1

Framework Version

2.110.1

Node.js Version

20.9.0

OS

WSL2 - Ubuntu 22.04.2 LTS

Language

TypeScript

Language Version

5.2.2

Other information

No response

[pahud]

Yes, this seems to be a bug.

[tmokmss]

You can try --exclusively flag; this should save some time by skipping bundling (not entire synthesis though) of stacks you didn't specified.

e.g. cdk synth --exclusively 'Dev/*'

[castleadmin]

You can try --exclusively flag; this should save some time by skipping bundling (not entire synthesis though) of stacks you didn't specified.

e.g. cdk synth --exclusively 'Dev/*'

Indeed, through the usage of the --exclusively flag only the Lambda function assets for the Dev environment are synthesized. This is a nice workaround, which speeds up local testing of Lambda functions with AWS SAM. The execution of the synthesize step is needed to generate the CloudFormation template and assets.

[AllanOricil]

@tmokmss

the -e flag does not work all the time.

I have a simple stack called route-53-stack that is used by another stack called instance-provisioning-stack. I asked cdk to synth route-53-stack with the -e flag, but instead of building only this stack, cdk decided to synthesize instance-provisioning-stack. Why?

Im using cdk 2.139.1 (build b88f959)

Once I ran cdk synth -e node-ready-instance-provisioning-route-53-stack, cdk decided to synth this other stack that uses the one I wanted to synth.

[AllanOricil]

edit: it is actually building all my stacks

[AllanOricil]

after I commented all stacks but the one I was interested in synth, cdk did what I needed

[tmokmss]

@AllanOricil
iirc with --exclusively flag, only bundling steps (e.g. bundling assets like lambda code) are skipped, while building steps (e.g. generating CFn template) always run. In my experience, bundling steps are usually the biggest contributor to the duration of synthesis, so skipping bundling greatly reduced synthesis time.

That said, I sometimes add an environment variable to entirelly include/exclude certain stacks, that do not have dependencies with each other, from bin/cdk.ts, e.g.

// bin/cdk.ts
if (process.env.ENV=='prod') {
  new SomeStack(app, "ProdStack");
} else {
  new SomeStack(app, "DevStack");
}

this allows you to synth only the required stacks by modifying the cdk deploy command:

 # build and deploy only DevStack
npx cdk deploy --all

 # build and deploy only ProdStack
ENV=prod npx cdk deploy --all

[AllanOricil]

So there is some kind of bug because the bundling steps are also running 😅
I thought it was an issue with 2.139, but 2.138 also has this issue.

[AllanOricil]

this is how the instance-provisioning-stack bundles its lambda functions, which the bundling is triggered whenever I try to deploy that simple route-53-stack. Could the usage of cdk.DockerImage.fromBuild be causing the bundling stage of the instance-provisioning-stack when synth the route-53-stack with the -e flag?

bundling: {
          sourceMap: true,
          nodeModules: ["winston", "@sentry/serverless"],
          externalModules: ["aws-sdk", "@aws-sdk"],
          user: "root",
          forceDockerBundling: true,
          dockerImage: cdk.DockerImage.fromBuild(
            DEFAULT_DOCKER_BUILD_IMAGES_PATH,
            {
              file: "ci.Dockerfile",
            },
          ),
          commandHooks: {
            beforeBundling(i: string, o: string): string[] {
              return ['echo "BEFORE BUNDLING"'];
            },
            beforeInstall(i: string, o: string): string[] {
              return ['echo "BEFORE INSTALL"'];
            },
            afterBundling(i: string, o: string): string[] {
              return [
                'echo "AFTER BUNDLING"',
                `cp ${i}/.sentryclirc .`,
                `sentry-cli sourcemaps inject --org journeylab --project create-route-53-records .`,
                `sentry-cli sourcemaps upload --org journeylab --project create-route-53-records .`,
                `rm .sentryclirc`,
              ];
            },
          },
        },
      },

[AllanOricil]

@tmokmss evidence to show that -e flag has a problem

As you can see, cdk triggered the bundling phase of a stack that I did not even asked it to deploy when running cdk deploy with -e. This vpc stack has dependencies, but they shouldnt be built if Im not deploying them and Im using -e

[tmokmss]

cdk.DockerImage.fromBuild builds a container image as soon as it is called, and --exclusively flag does not affect that behavior. Maybe you should run some of the long-running commands in beforeBundling command hook, instead of running them in Dockerfile. I think command hooks do not run when you exclude the stack with exclusively flag.

[comcalvi]

This looks like the same issue as #6743.

[comcalvi]

That issue has more upvotes, so I'm closing this one in favor of the other one - please continue the discussion there.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.