Skip to content

Latest commit

 

History

History
348 lines (250 loc) · 14.2 KB

CONTRIBUTING.md

File metadata and controls

348 lines (250 loc) · 14.2 KB
Raw

Contributing to the AWS Cloud Development Kit

Thanks for your interest in contibuting to the AWS CDK! ❤️

This document describes how to set up a development environment and submit your contributions. Please read it carefully and let us know if it's not up-to-date (even better, submit a PR with your corrections ;-)).

Pull Requests

  1. If there isn't one already, open an issue describing what you intend to contribute. It's usful to communicate in advance, because sometimes, someone is already working in this space, so maybe it's worth collaborating with them instead of duplicating the efforts.
  2. Work your magic. Here are some guidelines:
    • Every change requires a unit test
    • If you change APIs, make sure to update the module's README file
    • Try to maintain a single feature/bugfix per pull request. It's okay to introduce a little bit of housekeeping changes along the way, but try to avoid conflating multiple features. Eventually all these are going to go into a single commit, so you can use that to frame your scope.
  3. Create a commit with the proposed change changes:
    • Commit title and message (and PR title and description) must adhere to conventionalcommits.
    • The title must begin with feat(module): title, fix(module): title, reactor(module): title or chore(module): title.
    • Title should be lowercase.
    • No period at the end of the title.
    • Commit message should describe motivation. Think about your code reviewers and what information they need in order to understand what you did. If it's a big commit (hopefully not), try to provide some good entry points so it will be easier to follow.
    • Commit message should indicate which issues are fixed: fixes #<issue> or closes #<issue>.
    • Shout out to collaborators.
    • If not obvious (i.e. from unit tests), describe how you verified that your change works.
    • If this commit includes a breaking change, the commit message must end with a a single pragraph BREAKING CHANGE: description of what broke and how to achieve this beahvior now.
  4. Push to a fork or to a branch (naming convention: <user>/<feature-bug-name>)
  5. Submit a Pull Requests on GitHub and assign the PR for a review to the "awslabs/aws-cdk" team.
  6. Discuss review comments and iterate until you get at least one “Approve”. When iterating, push new commits to the same branch. Usually all these are going to be squashed when you merge to master. The commit messages should be hints for you when you finalize your merge commit message.
  7. Make sure to update the PR title/description if things change. The PR title/description are going to be used as the commit title/message and will appear in the CHANGELOG, so maintain them all the way throughout the process.
  8. Make sure your PR builds successfully (we have CodeBuild setup to automatically build all PRs)
  9. Once approved and tested, a maintainer will squash-merge to master and will use your PR title/description as the commit message.

Tools

The CDK is a big project, and, at the moment, all of the CDK modules are mastered in a single monolithic repository (uses lerna). There are pros and cons to this approach, and it's especially valuable to maintain integrity in the early stage of thr project where things constantly change across the stack. In the future we believe many of these modules will be extracted to their own repositories.

Another complexity is that the CDK is packaged using jsii to multiple programming languages. This means that when a full build is complete, there will be a version of each module for each supported language.

However, in many cases, you can probably get away with just building a portion of the project, based on areas that you want to work on.

We recommend that you use Visual Studio Code to work on the CDK. Be sure to install the tslint extension for it as well, since we have strict linting rules that will prevent your code from compiling, but with VSCode and this extension can be automatically fixed for you by hitting Ctrl-. when your cursor is on a red underline.

Main build scripts

The build process is divided into stages, so you can invoke them as needed:

  • install.sh: installs all external dependencies and symlinks internal dependencies (using lerna link).
  • build.sh: runs npm build and npm test in all modules (in topological order).
  • pack.sh: packages all modules to all supported languages and produces a dist/ directory with all the outputs (running this script requires that you installed the toolchains for all target languages on your system).

Partial build tools

There are also two useful scripts in the scripts directory that can help you build part of the repo:

  • scripts/buildup: builds the current module and all of its dependencies (in topological order).
  • scripts/builddown: builds the current module and all of its consumers (in topological order).

Useful aliases

You can also add a few useful aliases to your shell profile:

# runs an npm script via lerna for a the current module
alias lr='lerna run --stream --scope $(node -p "require(\"./package.json\").name")'

# runs "npm run build" (build + test) for the current module
alias lb='lr build'
alias lt='lr test'

# runs "npm run watch" for the current module (recommended to run in a separate terminal session):
alias lw='lr watch'

pkglint

The pkglint tool "lints" package.json files across the repo according to rules.ts.

To evaluate (and attempt to fix) all package linting issues in the repo, run the following command from the root of the repository (after boostrapping):

$ lerna run pkglint

You can also do that per package:

$ lr pkglint

Development Workflows

Full clean build

Clone the repo:

$ git clone git@github.com/awslabs/aws-cdk
$ cd aws-cdk

If you already have a local repo and you want a fresh build, run git clean -fdx from the root.

Install and build:

$ ./install.sh
$ ./build.sh

If you also wish to package to all languages, make sure you have all the [toolchains](#Toolchains] and now run:

$ ./pack.sh

Partial build

In many cases, you don't really need to build the entire project. Say you want to work on the @aws-cdk/aws-ec2 module:

$ ./install.sh
$ cd packages/@aws-cdk/aws-ec2
$ ../../../scripts/buildup

Quick Iteration

After you've built the modules you want to work on once, use lr watch for each module that you are modifying.

Watch the EC2 and IAM modules in a second terminal session:

$ cd packages/@aws-cdk/aws-ec2
$ lr watch & # runs in the background
$ cd packages/@aws-cdk/aws-iam
$ lr watch & # runs in the background

Code...

Now to test, you can either use lr test or invoke nodeunit directory (faster, since "test" will also build):

$ cd packages/@aws-cdk/aws-iam
$ nodeunit test/test.*.js
<BOOM>

Build Documentation Only

The CDK documentation source is hosted under ./docs/src. Module reference documentation is gathered after build from the dist/sphinx directory (generated by jsii from source via the ./pack.sh script).

To build the docs even if reference docs are not present:

$ cd docs
$ BUILD_DOCS_DEV=1 ./build-docs.sh

Dependencies

Adding Dependencies

The root package.json includes global devDependencies (see lerna docs) on the topic.

  • To add a global dependency, run npm i --save-dev <dep> at the root.
  • To add a dependency for a specific module, run npm i <dep> inside the module's directory.

Guidelines:

  • We cannot accept dependencies that use non-permissive open source licenses (Apache, MIT, etc).
  • Make sure dependencies are defined using caret ranges (e.g. ^1.2.3). This enables non-breaking updates to automatically be picked up.
  • Make sure package-lock.json files are included in your commit.

Finding Dependency Cycles

You can use find-cycles to print a list of internal dependency cycles:

$ scripts/find-cycles.sh
Cycle: @aws-cdk/aws-iam => @aws-cdk/assert => aws-cdk => @aws-cdk/aws-s3 => @aws-cdk/aws-kms => @aws-cdk/aws-iam
Cycle: @aws-cdk/assert => aws-cdk => @aws-cdk/aws-s3 => @aws-cdk/aws-kms => @aws-cdk/assert
Cycle: @aws-cdk/aws-iam => @aws-cdk/assert => aws-cdk => @aws-cdk/aws-s3 => @aws-cdk/aws-iam
Cycle: @aws-cdk/assert => aws-cdk => @aws-cdk/aws-s3 => @aws-cdk/assert
Cycle: @aws-cdk/assert => aws-cdk => @aws-cdk/aws-cloudformation => @aws-cdk/assert
Cycle: @aws-cdk/aws-iam => @aws-cdk/assert => aws-cdk => @aws-cdk/util => @aws-cdk/aws-iam
Cycle: @aws-cdk/aws-sns => @aws-cdk/aws-lambda => @aws-cdk/aws-codecommit => @aws-cdk/aws-sns
Cycle: @aws-cdk/aws-sns => @aws-cdk/aws-lambda => @aws-cdk/aws-codecommit => @aws-cdk/aws-codepipeline => @aws-cdk/aws-sns

Updating all Dependencies

We use npm update to

  1. Obtain a fresh clone from “master”
  2. Run ./install.sh and ./build.sh to make sure the current HEAD is not broken (should never be...).
  3. Once build succeeded, run: 
    $ npm update # to update the root deps
$ lerna exec npm update # to update deps in all modules
  4. This will probably install some new versions and update package.json and package-lock.json files.
  5. Now, run ./build.sh again to verify all tests pass.
  6. Submit a Pull Request.

Troubleshooting common issues

Most build issues can be solved by doing a full clean rebuild:

$ git clean -fqdx .
$ ./build.sh

However, this will be time consuming. In this section we'll describe some common issues you may encounter and some more targeted commands you can run to resolve your issue.

  • The compiler is throwing errors on files that I renamed/it's running old tests that I meant to remove/code coverage is low and I didn't change anything.

If you switch to a branch in which .ts files got renamed or deleted, the generated .js and .d.ts files from the previous compilation run are still around and may in some cases still be picked up by the compiler or test runners.

Run the following to clear out stale build artifacts:

$ scripts/clean-stale-files.sh
  • I added a dependency but it's not being picked up by the build

You need to tell Lerna to update all dependencies:

$ node_modules/.bin/lerna bootstrap
  • I added a dependency but it's not being picked up by a watch background compilation run.

No it's not. After re-bootstrapping you need to restart the watch command.

  • I added a dependency but it's not being picked up by Visual Studio Code (I still get red underlines).

The TypeScript compiler that's running has cached your dependency tree. After re-bootstrapping, restart the TypeScript compiler.

Hit F1, type > TypeScript: Restart TS Server.

  • I'm doing refactorings between packages and compile times are killing me/I need to switch between differently-verionsed branches a lot and rebuilds because of version errors are taking too long.

Our build steps for each package do a couple of things, such as generating code and generating JSII assemblies. If you've done a full build at least once to generate all source files, you can do a quicker TypeScript-only rebuild of the entire source tree by doing the following:

# Only works after at least one full build to generate source files
$ scripts/build-typescript.sh

# Also works to start a project-wide watch compile
$ scripts/build-typescript.sh -w

This does not do code generation and it does not do JSII checks and JSII assembly generation. Instead of doing a package-by-package ordered build, it compiles all .ts files in the repository all at once. This takes about the same time as it does to compile the biggest package all by itself, and on my machine is the difference between a 15 CPU-minute build and a 20 CPU-second build. If you use this methods of recompiling and you want to run the test, you have to disable the built-in rebuild functionality of lerna run test:

$ CDK_TEST_BUILD=false lr test

Toolchains

If you wish to use pack.sh to package the project to all supported languages, you will need the following toolchains installed:

Linking against this repository

The script ./link-all.sh can be used to generate symlinks to all modules in this repository under some node_module directory. This can be used to develop against this repo as a local dependency.

One can use the postinstall script to symlink this repo:

{
  "scripts": {
    "postinstall": "../aws-cdk/link-all.sh"
  }
}

This assumes this repo is a sibling of the target repo and will install the CDK as a linked dependency during npm install.

Adding Language Support

The CDK uses jsii to generate language bindings for CDK classes, which proxy interaction to a node.js child process in runtime.

To vend another language for the CDK (given there's jsii support for it):

  1. Create a directory packages/aws-cdk-xxx (where "xxx" is the language).
  2. Look at aws-cdk-java/package.json as a reference on how to setup npm build that uses pacmak to generate the code for all CDK modules and then compile and wrap the package up.
  3. Edit bundle-beta.sh and add CDK and jsii artifacts for your language under repo/xxx
  4. Add a cdk init template for your language (see packages/aws-cdk/lib/init-templates).
  5. Edit getting-started.rst and make there there's a getting started sections and examples for the new language.