Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pipelines): expose crossRegionReplicationBuckets #28447

Merged
merged 29 commits into from
Jan 8, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
27276f4
feat(pipelines): expose crossRegionReplicationBuckets
ahammond Dec 21, 2023
bf9068b
add README update
ahammond Dec 21, 2023
c9bd6a5
Merge branch 'main' into expose-crossRegionReplicationBuckets
ahammond Dec 21, 2023
7e797bf
Apply suggestions from code review
kaizencc Dec 23, 2023
c40df96
fix linting errors
jose-clickup Dec 29, 2023
377a595
added integ test for crossRegionReplicationBuckets prop
jose-clickup Dec 29, 2023
74b5a26
Merge pull request #69 from jose-clickup/add-CRR-buckets-integ-test
ahammond Jan 2, 2024
29f6e24
Merge branch 'main' into expose-crossRegionReplicationBuckets
ahammond Jan 2, 2024
322557e
Merge branch 'main' into expose-crossRegionReplicationBuckets
ahammond Jan 2, 2024
469604f
try using fromBucketAttributes for remote bucket
ahammond Jan 2, 2024
5a9a58f
update license
jose-clickup Jan 2, 2024
f456ed6
update integ test for newcodepipeline-with-cross-region-replication-b…
jose-clickup Jan 2, 2024
ac4d76a
Merge remote-tracking branch 'origin/expose-crossRegionReplicationBuc…
jose-clickup Jan 2, 2024
8ba35e6
added newpipeline-with-cross-region-replication-buckets integ-test sn…
jose-clickup Jan 2, 2024
31d4a76
remove comment
jose-clickup Jan 2, 2024
1cfa0af
update README
jose-clickup Jan 2, 2024
e2c7478
Merge pull request #70 from jose-clickup/add-CRR-buckets-integ-test
ahammond Jan 2, 2024
f885fc9
Update packages/aws-cdk-lib/pipelines/README.md
ahammond Jan 2, 2024
07f1da1
Merge branch 'main' into expose-crossRegionReplicationBuckets
kaizencc Jan 3, 2024
b2a055c
update unit test for crossRegionReplicationBuckets
jose-clickup Jan 3, 2024
f2e8d9b
Merge branch 'expose-crossRegionReplicationBuckets' into add-CRR-buck…
jose-clickup Jan 3, 2024
c208e0f
Merge pull request #71 from jose-clickup/add-CRR-buckets-integ-test
ahammond Jan 3, 2024
a4fe793
fixed README ts build including kms import
jose-clickup Jan 4, 2024
a1ee2f6
Merge branch 'expose-crossRegionReplicationBuckets' into add-CRR-buck…
jose-clickup Jan 4, 2024
8b122e3
Merge pull request #72 from jose-clickup/add-CRR-buckets-integ-test
ahammond Jan 4, 2024
3ea3f21
Merge branch 'main' into expose-crossRegionReplicationBuckets
ahammond Jan 4, 2024
7a8be3c
Update packages/aws-cdk-lib/pipelines/lib/codepipeline/codepipeline.ts
kaizencc Jan 4, 2024
0e50f0f
Merge branch 'main' into expose-crossRegionReplicationBuckets
kaizencc Jan 8, 2024
6f13109
Merge branch 'main' into expose-crossRegionReplicationBuckets
mergify[bot] Jan 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions packages/aws-cdk-lib/pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1232,6 +1232,49 @@ and orphan the old bucket. You should manually delete the orphaned bucket
after you are sure you have redeployed all CDK applications and there are no
more references to the old asset bucket.

## Considerations around Running at Scale

If you are planning to run pipelines for more than a hundred repos
deploying across multiple regions, then you will want to consider reusing
both artifacts buckets and cross-region replication buckets.

In a situation like this, you will want to have a separate CDK app / dedicatd repo which creates
kaizencc marked this conversation as resolved.
Show resolved Hide resolved
and managed the buckets which will be shared by the pipelines of all your other apps.
Note that this app must NOT be using the shared buckets because of chicken & egg issues.

The following code assumes you have created and are managing your buckets in the aforementioned
separate cdk repo and are just importing them for use in one of your (many) pipelines.

```typescript
kaizencc marked this conversation as resolved.
Show resolved Hide resolved
let sharedArtifactBucketArn: string;
let sharedArtifactKeyArn: string;

let sharedXRegionUsWest2BucketArn: string;
let sharedXRegionUsWest2KeyArn: string;

const artifactBucket = s3.Bucket.fromBucketAttributes(scope, 'bucketArn', {
bucketArn: sharedArtifactBucketArn,
encryptionKey: kms.Key.fromKeyArn(scope, 'keyArn', sharedArtifactKeyArn),
});

const usWest2Bucket = s3.Bucket.fromBucketAttributes(scope, 'us-west-2Bucket', {
bucketArn: sharedXRegionUsWest2BucketArn,
encryptionKey: kms.Key.fromKeyArn(scope, 'keyArn', sharedXRegionUsWest2KeyArn),
});

const crossRegionReplicationBuckets: Record<[key: string]: s3.IBucket> = {
'us-west-2': usWest2Bucket,
// Support for additional regions.
}

let otherProps: pipelines.CodePipelineProps;
const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
...otherProps,
// Use shared buckets.
artifactBucket,
crossRegionReplicationBuckets,
});
```
## Context Lookups

You might be using CDK constructs that need to look up [runtime
Expand Down
17 changes: 17 additions & 0 deletions packages/aws-cdk-lib/pipelines/lib/codepipeline/codepipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,19 @@ export interface CodePipelineProps {
* @default - A new S3 bucket will be created.
*/
readonly artifactBucket?: s3.IBucket;

/**
* A map of region to S3 bucket name used for cross-region CodePipeline.
* For every Action that you specify targeting a different region than the Pipeline itself,
* if you don't provide an explicit Bucket for that region using this property,
* the construct will automatically create a Stack containing an S3 Bucket in that region.
*
* Passed directly through to the {@link cp.Pipeline}.
*
* @default - None.
kaizencc marked this conversation as resolved.
Show resolved Hide resolved
*/

kaizencc marked this conversation as resolved.
Show resolved Hide resolved
readonly crossRegionReplicationBuckets?: { [region: string]: s3.IBucket };
}

/**
Expand Down Expand Up @@ -440,6 +453,9 @@ export class CodePipeline extends PipelineBase {
if (this.props.enableKeyRotation !== undefined) {
throw new Error('Cannot set \'enableKeyRotation\' if an existing CodePipeline is given using \'codePipeline\'');
}
if (this.props.crossRegionReplicationBuckets !== undefined) {
throw new Error('Cannot set \'crossRegionReplicationBuckets\' if an existing CodePipeline is given using \'codePipeline\'');
}
if (this.props.reuseCrossRegionSupportStacks !== undefined) {
throw new Error('Cannot set \'reuseCrossRegionSupportStacks\' if an existing CodePipeline is given using \'codePipeline\'');
}
Expand All @@ -455,6 +471,7 @@ export class CodePipeline extends PipelineBase {
this._pipeline = new cp.Pipeline(this, 'Pipeline', {
pipelineName: this.props.pipelineName,
crossAccountKeys: this.props.crossAccountKeys ?? false,
crossRegionReplicationBuckets: this.props.crossRegionReplicationBuckets,
reuseCrossRegionSupportStacks: this.props.reuseCrossRegionSupportStacks,
// This is necessary to make self-mutation work (deployments are guaranteed
// to happen only after the builds of the latest pipeline definition).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,18 @@ describeDeprecated('codepipeline existing', () => {
});
}).toThrow("Cannot set 'enableKeyRotation' if an existing CodePipeline is given using 'codePipeline'");
});

test('Does not allow setting crossRegionReplicationBuckets if an existing CodePipeline is given', () => {
const app = new cdk.App();
const stack = new cdk.Stack(app, 'PipelineStack');
const existingCodePipeline = new codePipeline.Pipeline(stack, 'CustomCodePipeline');

expect(() => {
new cdkp.CdkPipeline(stack, 'CDKPipeline', {
crossRegionReplicationBuckets: {}, // Even the empty set is forbidden.
codePipeline: existingCodePipeline,
cloudAssemblyArtifact: new codePipeline.Artifact(),
});
}).toThrow("Cannot set 'crossRegionReplicationBuckets' if an existing CodePipeline is given using 'codePipeline'");
});
});
Loading