Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pipelines): revised version of the API #12326

Merged
merged 89 commits into from
Jul 15, 2021
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
29ef96e
WIPWIP
rix0rrr Nov 23, 2020
ffd007a
Framework of new API
rix0rrr Dec 24, 2020
70fe76c
Adding Synth
rix0rrr Dec 24, 2020
342f590
CloudFormation actions
rix0rrr Dec 24, 2020
be89cb3
Don't need these anymore they just distract
rix0rrr Jan 4, 2021
bc4a3ce
Merge master into huijbers/tryout-pipelines-refactor
skinny85 Jan 6, 2021
54137d5
Cosmetic changes to make the package build.
skinny85 Jan 6, 2021
9811fc2
Making layers more clear
rix0rrr Jan 7, 2021
cea1787
Fix some bugs, show usage (already works for a good bit in CodePipeline)
rix0rrr Jan 8, 2021
ddcb77c
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jan 11, 2021
52fe73e
Starting to move tests over
rix0rrr Jan 13, 2021
27b53a6
Add open issues list, some additional CodeBuild configuration
rix0rrr Jan 13, 2021
1c43586
WIP
rix0rrr May 27, 2021
7f2971e
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr May 27, 2021
11b1f8e
Make it compile again
rix0rrr May 27, 2021
b9b1465
Make tests pass
rix0rrr May 31, 2021
91f15c1
Add additional build outputs
rix0rrr May 31, 2021
d79cca9
WIP
rix0rrr Jun 8, 2021
f09d81a
Still fixing tests
rix0rrr Jun 9, 2021
b98ca04
Make tests work
rix0rrr Jun 10, 2021
35a37d1
Rename more classes to XxxStep
rix0rrr Jun 10, 2021
3f3e2cd
Rename `build()` -> `buildPipeline()`
rix0rrr Jun 10, 2021
9160978
Fix incorrect forwarding of codebuildstep
rix0rrr Jun 10, 2021
2bfedfc
Make integ test pass
rix0rrr Jun 10, 2021
eb0a966
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jun 10, 2021
c475fbb
Support consuming stack outputs
rix0rrr Jun 11, 2021
ce2aa98
chore(pipelines): added abstractions for managing compliance between …
Jun 18, 2021
1bdec25
Started moving tests over to the compliance system
Jun 21, 2021
5be0cd8
Moved pipeline test
Jun 21, 2021
2ce4322
feat(pipelines): changes needed to support other engines (#15191)
Jun 24, 2021
d590a16
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jul 1, 2021
9440646
Reorganizing tests
rix0rrr Jul 1, 2021
f266f3b
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jul 1, 2021
d54dc06
Reorganizing test suite
rix0rrr Jul 1, 2021
0b9ef59
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jul 5, 2021
7fbd19e
Proper inheritance of CodeBuild project options
rix0rrr Jul 6, 2021
52c451b
Making a lot of tests pass
rix0rrr Jul 8, 2021
7146236
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jul 8, 2021
5fa2d9b
Passing buildSpec via file
rix0rrr Jul 8, 2021
ecbc7aa
Ported Docker Credentials feature
rix0rrr Jul 8, 2021
ddefa65
Remove `synth` defaults from pipeline
rix0rrr Jul 8, 2021
5b9c10b
Properly inherit VPC into CodeBuildStep.
rix0rrr Jul 8, 2021
53867cd
Add test to prove it
rix0rrr Jul 8, 2021
69d8b39
Logical ID stability
rix0rrr Jul 9, 2021
edd6565
Reduce duplication in factory usage
rix0rrr Jul 9, 2021
3838edc
Simplify CodeBuild handling
rix0rrr Jul 9, 2021
a47f45e
Add docstrings
rix0rrr Jul 12, 2021
bb3b830
New pipeline integ
rix0rrr Jul 12, 2021
545d11f
Merge remote-tracking branch 'origin/master' into huijbers/tryout-pip…
rix0rrr Jul 12, 2021
e0c1c33
Rename `synthStep` => `synth`
rix0rrr Jul 13, 2021
ca36515
Add README and some renames
rix0rrr Jul 13, 2021
89aed64
Pull the Construct thread a bit more
rix0rrr Jul 13, 2021
ecf75a0
Added codePipeline to CodePipelineEngineProps
Jul 13, 2021
7d2f006
GitHub branch is now mandatory
Jul 13, 2021
fe6f1a0
Revert "GitHub branch is now mandatory"
Jul 13, 2021
60b3b21
Review comments
rix0rrr Jul 13, 2021
09f8ba3
Merge branch 'huijbers/tryout-pipelines-refactor' of github.com:aws/a…
rix0rrr Jul 13, 2021
6efc354
Revert "Pull the Construct thread a bit more"
rix0rrr Jul 13, 2021
3ddeb8f
Update CodePipelineActionFactory signature
rix0rrr Jul 13, 2021
1a19702
Missing saves
rix0rrr Jul 13, 2021
3bfdd63
Missed refactors
rix0rrr Jul 13, 2021
c0517d7
Revert optionality in Record types
rix0rrr Jul 13, 2021
02c4161
Markdownlint
rix0rrr Jul 13, 2021
3841dd8
Collapse 'Blueprint' <-> 'Pipeline', 'CodePipeline' <-> 'CodePipeline…
rix0rrr Jul 13, 2021
4fb5af4
Revert "Revert optionality in Record types"
rix0rrr Jul 14, 2021
fd8d095
Hide PipelineQueries from the public interface
rix0rrr Jul 14, 2021
9cacf3f
Pipeline => PipelineBase
rix0rrr Jul 14, 2021
bcef6ab
Missing commit
rix0rrr Jul 14, 2021
1cfbb4b
Revert optionality in Record types
rix0rrr Jul 13, 2021
185f2c4
Silence linter
rix0rrr Jul 14, 2021
7b1ed04
Remove SynthStep
rix0rrr Jul 14, 2021
cd57bea
Escape hatching: integrating with existing pipelines
rix0rrr Jul 14, 2021
c7a3a47
Rename `github` back to `gitHub`, for consistency with all other cons…
rix0rrr Jul 14, 2021
c74d351
New sources: S3, CodeStar and CodeCommit
Jul 14, 2021
7459c4b
Add some missing tests
rix0rrr Jul 14, 2021
5f7c074
Merge branch 'huijbers/tryout-pipelines-refactor' of github.com:aws/a…
rix0rrr Jul 14, 2021
62355a6
primaryOutput made mandatory
Jul 14, 2021
c752654
Must rename internal field
rix0rrr Jul 14, 2021
ddd6330
Fix up linter and typescript issues
rix0rrr Jul 14, 2021
382183d
Add periods.
rix0rrr Jul 14, 2021
fbb86af
branch made a positional parameter where applicable
Jul 14, 2021
4f84e40
Update README
rix0rrr Jul 14, 2021
926a857
ScriptStep -> ShellStep
rix0rrr Jul 14, 2021
0f824b5
Merge branch 'huijbers/tryout-pipelines-refactor' of github.com:aws/a…
rix0rrr Jul 14, 2021
83a2b41
Add `dockerEnabledForSynth` flag
rix0rrr Jul 14, 2021
6bcbe53
Added more information about .gitHub() and .codeStarConnection() in t…
Jul 14, 2021
b7c2603
Doc improvements
Jul 14, 2021
536a81f
Final source tweaks
rix0rrr Jul 15, 2021
3b3e1c0
Merge branch 'master' into huijbers/tryout-pipelines-refactor
mergify[bot] Jul 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions packages/@aws-cdk/aws-codepipeline/lib/pipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -479,7 +479,7 @@ export class Pipeline extends PipelineBase {
}

/** @internal */
public _attachActionToPipeline(stage: Stage, action: IAction, actionScope: CoreConstruct): FullActionDescriptor {
public _attachActionToPipeline(stage: Stage, action: IAction, actionScope: Construct): FullActionDescriptor {
const richAction = new RichAction(action, this);

// handle cross-region actions here
Expand All @@ -491,8 +491,8 @@ export class Pipeline extends PipelineBase {
// // CodePipeline Variables
validateNamespaceName(richAction.actionProperties.variablesNamespace);

// bind the Action
const actionConfig = richAction.bind(actionScope, stage, {
// bind the Action (type h4x)
const actionConfig = richAction.bind(actionScope as CoreConstruct, stage, {
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
role: actionRole ? actionRole : this.role,
bucket: crossRegionInfo.artifactBucket,
});
Expand Down
11 changes: 10 additions & 1 deletion packages/@aws-cdk/aws-codepipeline/lib/private/stage.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import * as events from '@aws-cdk/aws-events';
import * as cdk from '@aws-cdk/core';
import { Construct, Node } from 'constructs';
import { IAction, IPipeline, IStage } from '../action';
import { Artifact } from '../artifact';
import { CfnPipeline } from '../codepipeline.generated';
Expand Down Expand Up @@ -137,7 +138,15 @@ export class Stage implements IStage {

private attachActionToPipeline(action: IAction): FullActionDescriptor {
// notify the Pipeline of the new Action
const actionScope = new cdk.Construct(this.scope, action.actionProperties.actionName);
//
// It may be that a construct already exists with the given action name (CDK Pipelines
// may do this to maintain construct tree compatibility between versions).
//
// If so, we simply reuse it.
let actionScope = Node.of(this.scope).tryFindChild(action.actionProperties.actionName) as Construct | undefined;
if (!actionScope) {
actionScope = new cdk.Construct(this.scope, action.actionProperties.actionName);
}
return this._pipeline._attachActionToPipeline(this, action, actionScope);
}

Expand Down
15 changes: 15 additions & 0 deletions packages/@aws-cdk/pipelines/lib/blueprint/asset-type.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/**
* Type of the asset that is being published
*/
export enum AssetType {
/**
* A file
*/
FILE = 'file',

/**
* A Docker image
*/
DOCKER_IMAGE = 'docker-image',
}

93 changes: 93 additions & 0 deletions packages/@aws-cdk/pipelines/lib/blueprint/blueprint.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import * as cdk from '@aws-cdk/core';
import { Step } from './step';
import { AddStageOpts, Wave } from './wave';

/**
* Construction properties for a `Blueprint`.
*/
export interface BlueprintProps {
/**
* The build step that produces the CDK Cloud Assembly
*
* The primary output of this step needs to be the `cdk.out` directory
* generated by the `cdk synth` command.
*
* If you use `SynthStep`, the output directory will automatically be
* configured correctly; if you use a different kind of `Step`, you
* may need to configure it explicitly.
*/
readonly synthStep: Step;
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* A description of a set of application deployments
*
* You don't need to create an instance of `Blueprint` yourself;
* one will be automatically created for you when you instantiate
* a `Pipeline`.
*/
export class Blueprint {
/**
* The build step that produces the CDK Cloud Assembly
*/
public readonly synthStep: Step;

/**
* The waves in this pipeline
*/
public readonly waves: Wave[];

constructor(props: BlueprintProps) {
this.synthStep = props.synthStep;
this.waves = [];
}

/**
* Add a Wave to the pipeline, for deploying multiple Stages in parallel
*
* Use the return object of this method to deploy multiple stages in parallel.
*
* Example:
*
* ```ts
* const wave = pipeline.addWave('MyWave');
* wave.addStage(new MyStage('Stage1', ...));
* wave.addStage(new MyStage('Stage2', ...));
* ```
*/
public addWave(id: string, options: AddWaveOptions = {}) {
const wave = new Wave(id, options);
this.waves.push(wave);
return wave;
}

/**
* Deploy a single Stage by itself
*
* Add a Stage to the pipeline, to be deployed in sequence with other
* Stages added to the pipeline. All Stacks in the stage will be deployed
* in an order automatically determined by their relative dependencies.
*/
public addStage(stage: cdk.Stage, options: AddStageOpts = {}) {
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
return this.addWave(stage.stageName).addStage(stage, options);
}
}

/**
* Options to pass to `addWave`
*/
export interface AddWaveOptions {
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
/**
* Additional steps to run before any of the stages in the wave
*
* @default - No additional steps
*/
readonly pre?: Step[];

/**
* Additional steps to run after all of the stages in the wave
*
* @default - No additional steps
*/
readonly post?: Step[];
}
66 changes: 66 additions & 0 deletions packages/@aws-cdk/pipelines/lib/blueprint/file-set.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import { Step } from './step';

/**
* A set of files traveling through the deployment pipeline
*
* Individual steps in the pipeline produce or consume
* `FileSet`s.
*/
export class FileSet implements IFileSetProducer {
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
/**
* The primary output of a file set producer
*
* The primary output of a FileSet is itself.
*/
public readonly primaryOutput?: FileSet = this;
private _producer?: Step;

constructor(
/** Identifier for this file set */
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
public readonly id: string, producer?: Step) {
this._producer = producer;
}

/**
* The Step that produces this FileSet
*/
public get producer() {
if (!this._producer) {
throw new Error(`FileSet '${this.id}' doesn\'t have a producer; call 'fileSet.producedBy()'`);
}
return this._producer;
}

/**
* Mark the given Step as the producer for this FileSet
*
* This method can only be called once.
*/
public producedBy(producer?: Step) {
if (this._producer) {
throw new Error(`FileSet '${this.id}' already has a producer (${this._producer}) while setting producer: ${producer}`);
}
this._producer = producer;
}

/**
* Return a string representation of this FileSet
*/
public toString() {
return `FileSet(${this.id})`;
}
}

/**
* Any class that produces, or is itself, a `FileSet`
*
* Steps implicitly produce a primary FileSet as an output.
*/
export interface IFileSetProducer {
/**
* The `FileSet` produced by this file set producer
*
* @default - This producer doesn't produce any file set
*/
readonly primaryOutput?: FileSet;
}
11 changes: 11 additions & 0 deletions packages/@aws-cdk/pipelines/lib/blueprint/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export * from './asset-type';
export * from './blueprint';
export * from '../helpers-internal/blueprint-queries';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is intentional that we are exporting internal stuff?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it shouldn't be exported twice, obviously. But BlueprintQueries is part of the public API surface.

export * from './file-set';
export * from './script-step';
export * from './stack-deployment';
export * from './stage-deployment';
export * from './step';
export * from './synth-step';
export * from './wave';
export * from './manual-approval';
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved
40 changes: 40 additions & 0 deletions packages/@aws-cdk/pipelines/lib/blueprint/manual-approval.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { FileSet } from './file-set';
import { Step } from './step';

/**
* Construction properties for a `ManualApprovalStep`
*/
export interface ManualApprovalStepProps {
/**
* The comment to display with this manual approval
*
* @default - No comment
*/
readonly comment?: string;
}

/**
* A manual approval step
*
* If this step is added to a Pipeline, the Pipeline will
* be paused waiting for a human to resume it
*
* Only engines that support pausing the deployment will
* support this step type.
*/
export class ManualApprovalStep extends Step {
public readonly primaryOutput?: FileSet | undefined;
rix0rrr marked this conversation as resolved.
Show resolved Hide resolved

/**
* The comment associated with this manual approval
*
* @default - No comment
*/
public readonly comment?: string;

constructor(id: string, props: ManualApprovalStepProps = {}) {
super(id);

this.comment = props.comment;
}
}
Loading