Programmatic access of AWS CDK CLI

[hassankhan]

Description

Hi there, thanks for creating this package! I was just wondering if there's any way of triggering CDK CLI commands programatically? This would really help with respect to integrating with existing tools and workflows.

RFC 300: Programmatic access of AWS CDK CLI

Roles

Role	User
Proposed by	@hassankhan, @mrgrain
Author(s)	@mrgrain
API Bar Raiser	@rix0rrr
Stakeholders

See RFC Process for details

Workflow

• Tracking issue created (label: status/proposed)
• API bar raiser assigned (ping us at #aws-cdk-rfcs if needed)
• Kick off meeting
• RFC pull request submitted (label: status/review)
• Community reach out (via Slack and/or Twitter)
• API signed-off (label api-approved applied to pull request)
• Final comments period (label: status/final-comments-period)
• Approved and merged (label: status/approved)
• Execution plan submitted (label: status/planning)
• Plan approved and merged (label: status/implementing)
• Implementation complete (label: status/done)

---

Author is responsible to progress the RFC according to this checklist, and
apply the relevant labels to this issue so that the RFC table in README gets
updated.

[rix0rrr]

Hi @hassankhan,

The aws-cdk NodeJS package does expose some library functions that can be used from JavaScript/TypeScript projects directly. However, it's not a polished experience at the moment, because our attention has been focused more towards the construct library so far. You're very welcome to try it and tell us if you run into any issues, though. As a last resort, you can always shell out to the command-line tool.

I'm also curious: could you tell us something about the kinds of integration that you're thinking of?

[hassankhan]

Hi @rix0rrr,

Thanks for the info! In an ideal world, I'd love to see something like:

import CDK from '@aws-cdk/cdk';

import MyApp from './MyApp';

const cdk = new CDK({ app: '', versionReporting: false });

cdk.deploy(new MyApp())
  .then(() => {
    return remove(new App());
  })

With regards to integration, essentially, our aim was to define resources such as Lambdas and DynamoDB tables once in our application code, and generate valid snippets from them and insert them into a template for deployment. For example, a theoretical DynamoDB model would look something like this (using dynamodb-data-mapper):

@table('pets')
class Pet {
    @hashKey()
    id: string;

    @rangeKey({defaultProvider: () => new Date()})
    createdAt: Date;

    @attribute()
    isCat?: boolean;
}

The table schema would then be read from this class as part of the deployment process, converted into its constituent Construct and added to the App/Stack. We could use the same approach for other resource types such as SNS Topics and Lambda functions.

[rix0rrr]

Okay, thanks. I definitely see the value of the first one, especially in defining one-off tasks like EMR jobs or integ tests.

For the second one: you can already do this today, even without tool integration.

If you define you data model in a library, you can use this library both in your runtime code as well as introspect it in your infrastructure code. Your build step would produce both a CloudFormation template as well as a JAR, and during deployment you deploy both together.

[rui-ktei]

+1
I mentioned this in a duplicate ticket as well.

[BDQ]

I'd like this too, in my case I'd like to wrap a deploy of an ECS cluster so I can scale it manually after the deploy is done (I create it with desiredCount: 0 so the CloudFormation creating is quicker).

[dsilvasc]

Additional use cases from aws/aws-cdk#1133 - synthesizing stacks from an IDE debugger and embedding into JVM build systems (like Gradle) without having to shell out to a CLI that depends on node.js.

[mmoulton]

My team is currently programmatically invoking CDK to support dynamic composition of constructs to create stacks based on the answers to question we ask in a custom CLI tool we wrote. We are currently primarily using it to spin up either SAM apps, or static sites using codepipeline, codebuild, rds, vpc, ssm params, s3 buckets, cloudfront, certs, zones, etc.

I'd very much like an officially supported way of accomplishing this. Currently we are doing some very dirty things to mimic the nature of the CDK CLI and the fact it executes the cdk app multiple times to meet dependencies.

If the CDK team does not have plans to officially support this, I'd very much appreciate not making it harder to achieve this as the notes in aws/aws-cdk#2044 and aws/aws-cdk#2016 suggest.

[KarthickEmis]

Is there anyway of triggering AWS CDK ClI (CDK synth , CDK deploy) commands pro-grammatically in typescript ?

[auser]

I found a way to handle this. I'll create an example repo (hopefully) to demonstrate how I accomplished this process, but the short version looks like this:

Create an AppStack instance and set the synthesizer to the output of app.run():

import { AppStacks } from 'aws-cdk/lib/api/cxapp/stacks'
import { Configuration } from 'aws-cdk/lib/settings';
// ...
const out = this._app.run();        
const argv = {
    '$0': this._argv['$0'],
    '_': this._argv['_']
}
const configuration = new Configuration(argv)
await configuration.load()

const appStacks = new AppStacks({
    aws: this.sdk,
    configuration: configuration,
    synthesizer: async () => out
})

Using the AppStack instance, create an instance of the CdkToolkit:

const appStacks = await this.getAppStacks();

const provisioner = new CloudFormationDeploymentTarget({
    aws: this.sdk,
})
const cli = new CdkToolkit({ appStacks, provisioner });

In order to get the credentials, we'll need to muck around at a low-level with the AWS-SDK:

import { debug } from '@aws-rocket/core';
import AWS = require('aws-sdk')
import { SDK } from 'aws-cdk/lib/api/util/sdk';
import { Mode } from 'aws-cdk/lib/api/aws-auth/credentials'

async function getCredentialsConfig(sdk: SDK): Promise<any> {
    const region = await sdk.defaultRegion()
    const defaultAccount = await sdk.defaultAccount()
    const credentials = await (sdk as any).credentialsCache.get(defaultAccount, Mode.ForReading)
    
    return {
      region,
      credentials
    }
  }

export const awsCredentialsMiddleware = async (argv: any) => {
    debug(`Setting profile to: ${argv.profile}`)
    if (argv.profile) {
        const sdk = new SDK({
            profile: argv.profile
        })
        const credentials = await getCredentialsConfig(sdk)
        AWS.config.update(credentials)
        AWS.config.setPromisesDependency(Promise);
        argv.AWS = AWS
        argv.SDK = sdk
    }
    return argv
}

Finally, this all comes together using the actual deploy like so:

const appStacks = await this.getAppStacks();
const allStacks = await appStacks.listStacks()
const allStackNames = allStacks
    .map((s: cxapi.CloudFormationStackArtifact) => s.name)

const cli = await this.getCdkToolkit()
try {
    const res = await cli.deploy({
        stackNames: allStackNames,
        // `requireApproval` is so that the user never needs to 
        // accept anything for the deploy to actually occur
        requireApproval: RequireApproval.Never,
    })
    console.log('deploy result ->', res);
} catch (e) {
    console.error(`Error deploying`, e);
    process.exit(-1)
}

Hope this helps and hopfully this becomes a part of the actual library. I'll try to submit a PR if I can, but no promises. If anyone else has any better way of handling this, I'd love suggestions.

[jd-carroll]

This is all very interesting and hopeful that it will get implemented 🤞

@auser - Were you ever able to put together a repo / gist with all of this information? I follow the logic but I'm not sure how I follow how to assemble all of the pieces. Is each code snippet supposed to be in the same file and if so does each code section follow one an other? I can guess, but I also don't know where the this._app and this._argv come from.

@fulghum / @shivlaks - It looks like there is some movement towards an implementation. Do feel programmatic access might be limited in some areas or is the goal to have a full API access the same you would have on command line?

More or less, do you think something like the following would be supported:

import AWS from 'aws-sdk';
import { deploy } from 'aws-cdk';
import MyStack from './my-stack';

const cfnOutput = await deploy(new MyStack());
console.log(`Successfully deployed stack:\n${cfnOutput}`);

const s3FileAndDetails = { /* ... */ };
const s3 = new AWS.S3();
await s3.putObject(s3FileAndDetails).promise();

Also, instead of needing to run the configuration like @auser has above:

const argv = {
    '$0': this._argv['$0'],
    '_': this._argv['_']
}
const configuration = new Configuration(argv)
await configuration.load()

It would be great if we could skip the command line altogether and just pass in a json object with relevant configuration. (Thereby eliminating the need for a cdk.json, but one could exist with overrides)

// Use factory function to allow reading config overrides in async
const configuration = await Configuration.create({ /* json from cdk.json */ })

[ryan-mars]

If you want to synth programmatically to the directory of your choice...

// Let's say you want it synthed to ./build
const outdir = path.join(process.cwd(), 'build') 
// App's constructor let's you specify the outdir 
const app = new cdk.App({outdir})
new ApiLambdaCrudDynamoDBStack(app, 'ApiLambdaCrudDynamoDBExample')
app.synth()

If you want to inspect what was produced app.synth() returns a CloudAssembly which can be interrogated with getStack('your stack name')

I haven't tried deploying programmatically yet.

[jd-carroll]

When implementing this feature, would it make sense to do it in the context of aws-sdk-js-v3?
https://github.com/aws/aws-sdk-js-v3

[ralovely]

(One of) the premise of CDK being "programmatic CloudFormation", it opens the way for fully programmable infrastructure, potentially even starting with the account creation
(and that's awesome, we (building an "Ops as a Service" product) have been hoping for this for quite some time, even looking into developing our own solution).
Creating a full, real-life infrastructure requires more features than CloudFormation can offers, even considering CustomResources.
Some of those will require the SDK ; some others need to talk to 3rd party services ; fetch data before ; output/notify after.

Being able to use CDK not so much as a toolkit/cli, not as a complete solution, but as a lib, as a way to integrate the infrastructure-control layer into a programm would be great. As great as CDK promise to be, it's not a complete solution, probably can't, and shouldn't even try to be one.
Of course, one can always shell out, wrap around CDK CLI, but that's not ideal.

It ties in with a few other GitHub issues:

• Support for app.postDeploy() for post-deploy functions
• CLI: way to get outputs from deployed stack
• Stacks and/or Contructs should emit events
• Document how to use stack parameters (to some degree)

[jfaixo]

Another use case : when your CDK stack creates Cognito UserPool Client Apps, it generates client id and secrets that need to be injected in your clients. If your client is a webapp in S3, that means you have to do steps in this order:

• create userpool and clients, get id values
• build webapp with updated id values
• deploy webapp to S3

If it was possible to deploy programmatically, it could be possible to have the whole process written in the same tool (for example nodejs). Otherwise (as of now I do this way), you have to rely to some other external tool to call the CLI multiple times and schedule these actions (like a bash script, or other nodejs script through process exec..).

[abierbaum]

I just found a nice little helper that wraps some of this here: https://github.com/dmitrii-t/cdk-util May be worth a look for people that want to do this type of thing. Definitely can't wait to see it in the core API with first class support.

[ryan-mars]

@jfaixo

If it was possible to deploy programmatically

It is.

import * as s3 from "@aws-cdk/aws-s3";
import * as s3deploy from "@aws-cdk/aws-s3-deployment"; // <<--- this right here
import * as cdk from "@aws-cdk/core";

export class WebStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const websiteBucket = new s3.Bucket(this, "WebsiteBucket", {
      websiteIndexDocument: "index.html",
      publicReadAccess: true,
      versioned: true
    });

    new s3deploy.BucketDeployment(this, "DeployWebsite", {
      sources: [s3deploy.Source.asset("./build")], // <<--- folder to upload
      destinationBucket: websiteBucket
    });
  }
}

[abierbaum]

Unfortunately I have found that the code and helpers I can find online are all out of date with the latest version of the CDK. How are people using the CDK in the case where you have a command line tool that needs to dynamically allocate and manage resources on AWS?

[sukhwant]

I am looking to use it from JAVA.

[zyrorl]

I've been able to sort of get this working programatically but have discovered that the cli.deploy etc functions return void, undefined, boolean and 0/1 values instead of useful values like cloudformation stack arns, outputs etc.

[shivlaks]

from @benwainwright

Currently, I can get a list of stacks that will be generated by my project in the order they should be deployed (this is important; see below) by running the command cdk list
It would be useful if the output from this command could be exposed as an api in a way that doesn't require full synthesis of the templates under the hood.

full use case in aws/aws-cdk#8436

[Guitarkalle]

This would be wonderful. Now it feels like CI integration is a pain.. You have to parse output for changeset names and stack ids after deploying to do any more serious CI flow. Using the synthesizing & deploy parts of the tool programmatically and also having the CLI able to output in a JSON format instead of just console logs with necessary information would be perfect

[moshir]

Hi,
any update on this one ?
I'm trying to automate the creation of AWS resources from a lambda function, and would be great to have a programmatic API in addition to the cdk CLI.
Alternatively, is there any way to simply get the cloudformation template string generated by synth(), and then pass it to boto3 cloudformation API ?

[rossng]

This would be very useful for my use case: dynamically deploying MediaLive broadcast pipelines. I don't really see shelling out to the CLI as a viable option.

[ignaloidas]

This would also be very useful for us for running integration tests. We currently shell out to CDK CLI for this, but that means we have to do some annoying things like having a separate app in test folder that adds more outputs, etc. Ability to deploy stacks with programmatic usage would help greatly for our use case.

[mrgrain]

Thanks all for the feedback so far. We have now released a first iteration of this: @aws-cdk/cli-lib-alpha

Please let me know what you're thinking and keep posting feature ideas. We are well aware this isn't a complete and ready solution yet (hence the experimental state), but it's an important stepping stone to gather interest and feedback.

[louisdussarps]

We may be a little late following previous AWS Annoucement.
Still, please allow me to present how we programmatically manage the CDK via Orbits (an openSource flow for DevOps).
It seems to me that Orbits' approch is still relevant if you have complex interactions to manage within your stacks or through a multi - steps deployment.

https://github.com/LaWebcapsule/orbits
https://github.com/LaWebcapsule/orbits/tree/main/src/examples/git-cdk-s3

[bastosvinicius]

Hello @mrgrain, I'm trying to implement this using Python 3.11, but actually facing issue related below when running synth:

cli.synth()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vinicius/.cache/pypoetry/virtualenvs/auk-I98UnL2N-py3.11/lib/python3.11/site-packages/aws_cdk/cli_lib_alpha/__init__.py", line 2276, in synth
    return typing.cast(None, jsii.ainvoke(self, "synth", [options]))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vinicius/.cache/pypoetry/virtualenvs/auk-I98UnL2N-py3.11/lib/python3.11/site-packages/jsii/_kernel/__init__.py", line 149, in wrapped
    return _recursize_dereference(kernel, fn(kernel, *args, **kwargs))
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vinicius/.cache/pypoetry/virtualenvs/auk-I98UnL2N-py3.11/lib/python3.11/site-packages/jsii/_kernel/__init__.py", line 457, in ainvoke
    promise = self.provider.begin(
              ^^^^^^^^^^^^^^^^^^^^
  File "/home/vinicius/.cache/pypoetry/virtualenvs/auk-I98UnL2N-py3.11/lib/python3.11/site-packages/jsii/_kernel/providers/process.py", line 387, in begin
    return self._process.send(request, BeginResponse)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vinicius/.cache/pypoetry/virtualenvs/auk-I98UnL2N-py3.11/lib/python3.11/site-packages/jsii/_kernel/providers/process.py", line 327, in send
    self._process.stdin.flush()
BrokenPipeError: [Errno 32] Broken pipe

My code is:

Running my click CLI command:

auk cloud aws instance create

Click method:

@aws.command('instance')
@click.pass_context
@click.argument('create', nargs=-1)
def instance(ctx, create: bool = False):
    print('create instances')
    if create:
        producer = Producer()
        cli = AwsCdkCli.from_cloud_assembly_directory_producer(producer)
        cli.synth(
            stacks=["EC2Stack"]
        )

aws.py file:

import os.path

from aws_cdk import Stack, App
from aws_cdk import aws_ec2 as ec2
from aws_cdk import aws_iam as iam
from aws_cdk.aws_s3_assets import Asset
from constructs import Construct
from aws_cdk.cli_lib_alpha import ICloudAssemblyDirectoryProducer
import jsii

dirname = os.path.dirname(__file__)


class EC2InstanceStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        vpc = ec2.Vpc(self, "VPC",
                      nat_gateways=0,
                      subnet_configuration=[ec2.SubnetConfiguration(
                          name="public", subnet_type=ec2.SubnetType.PUBLIC)]
                      )

        amzn_linux = ec2.MachineImage.latest_amazon_linux(
            generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
            edition=ec2.AmazonLinuxEdition.STANDARD,
            virtualization=ec2.AmazonLinuxVirt.HVM,
            storage=ec2.AmazonLinuxStorage.GENERAL_PURPOSE
        )

        role = iam.Role(self, "InstanceSSM",
                        assumed_by=iam.ServicePrincipal("ec2.amazonaws.com"))

        role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name(
            "AmazonSSMManagedInstanceCore"))

        instance = ec2.Instance(self, "Instance",
                                instance_type=ec2.InstanceType("t3.nano"),
                                machine_image=amzn_linux,
                                vpc=vpc,
                                role=role
                                )

        # # Script in S3 as Asset
        # asset = Asset(self, "Asset", path=os.path.join(
        #     dirname, "configure.sh"))
        # local_path = instance.user_data.add_s3_download_command(
        #     bucket=asset.bucket,
        #     bucket_key=asset.s3_object_key
        # )

        # # Userdata executes script from S3
        # instance.user_data.add_execute_file_command(
        #     file_path=local_path
        # )
        # asset.grant_read(instance.role)


@jsii.implements(ICloudAssemblyDirectoryProducer)
class Producer:
    def produce(self, context):
        app = App(context=context)
        # stack = Stack(app)
        stack = EC2InstanceStack(app, "EC2Stack")
        return app.synth().directory

Could you explain better how can I use this?

[bastosvinicius]

Following doc: https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.cli_lib_alpha/README.html

Versions:

[tool.poetry.dependencies]
python = "^3.11"
google-cloud-compute = "^1.11.0"
click = "^8.1.3"
aws-cdk-lib = "^2.82.0"
aws-cdk-cli-lib-alpha = "^2.82.0a0"
jsii = "^1.82.0"

[mrgrain]

Hey @bastosvinicius Apologies for this, but unfortunately Python is currently not supported by this alpha module. While there are some scenarios where it seems to work, most don't. This is due to an issue with jsii and how async code is handled. =/

[bastosvinicius]

Thanks for reply @mrgrain. Is there a forecast for definitive python support?

[mrgrain]

It's pretty much this PR: aws/jsii#3919
I was busy with other project work, hoping to get back to it in July.

[bastosvinicius]

I'll follow the PR and wait for the new version of jsii to be released to test it. Thanks for the feedback.

[evgenyka]

Should include other CLI commands, like cdk diff in the scope - aws/aws-cdk#679

[dreamorosi]

Hi, we use this package as part of our test harness to programmatically deploy infrastructure resources.

We are running into issues with Dependabot not picking up the package due to it being still in alpha, which causes issues with automatic updates for the rest of CDK libraries.

We are pursuing a fix on Dependabot side, but we'd be interested in also understanding what's the progress on this package becoming stable and being promoted in the main aws-cdk-lib/aws-cdk.

Is there any update that you could share?

[jiayiwang7]

Hi, we use this package as part of our test harness to programmatically deploy infrastructure resources.

We are running into issues with Dependabot not picking up the package due to it being still in alpha, which causes issues with automatic updates for the rest of CDK libraries.

We are pursuing a fix on Dependabot side, but we'd be interested in also understanding what's the progress on this package becoming stable and being promoted in the main aws-cdk-lib/aws-cdk.

Is there any update that you could share?

Hi, this feature is on our roadmap. We are currently finalizing the requirements / use cases before moving to the design phase. Will provide an update once available. Thanks!

[dreamorosi]

@jiayiwang7, I'm confused, maybe I got the wrong RFC.

We are already using the @aws-cdk/cli-lib-alpha package which has been in alpha for the past year.

Is this the same topic? My previous message was to ask whether there were plans to promote it to stable.

[AntonioAngelino]

We've been using it for years. We bootstrapped hundreds of AWS Accounts thanks to the lib and AWS Organizations.

[mrgrain]

@jiayiwang7, I'm confused, maybe I got the wrong RFC.

We are already using the @aws-cdk/cli-lib-alpha package which has been in alpha for the past year.

Is this the same topic? My previous message was to ask whether there were plans to promote it to stable.

Yes, same topic. We are looking in the missing requirements for stabilization. In it's current state, we don't consider it ready (e.g. no control over output, no return values, no useful exceptions).
We are also investigating if a standalone package makes more sense. And if so, we can likely deliver new features more incrementally under a more semver-like versioning (which I think would solve your issue with Dependabot).

[dreamorosi]

Thanks for your answer @mrgrain.

That would definitely solve our issues with versioning, and from a customer perspective having it separate even in its current state with a 0.x would make sense.

The only comment about this that I have is to ensure that when it comes to declaring aws-cdk-lib as peer dependency, ideally the ranges would be wide.

Otherwise we'll get into old issues like we had with CDK v1, which was constantly complaining about the slightest version mismatch, even between patch releases.

[mrgrain]

The only comment about this that I have is to ensure that when it comes to declaring aws-cdk-lib as peer dependency, ideally the ranges would be wide.

Yes, good feedback. Let's aim for ^2 as peer dependency requirement if we even need one.