-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(cdk): aspect framework and tag implementation #1451
Conversation
packages/@aws-cdk/aws-ec2/lib/vpc.ts
Outdated
this.tags.setTag(NAME_TAG, this.path, { overwrite: false }); | ||
// set first and let initial tags overwrite | ||
this.apply(new cdk.TagAspect(NAME_TAG, this.path)); | ||
const tags = props.tags || {}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wasn't the whole idea if the aspect system that we wouldn't need to explicitly support tags in each construct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This being a draft, I had left this for discussion about how much of a hard breaking change we want. I can remove all instances of tags
and TagManager
if we think this design is correct. I didn't want to overwrite the current PR just yet as I felt @rix0rrr and I were not quite on the same page yet.
} | ||
} | ||
|
||
export class RemoveTag extends TagBaseAspect { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably should have an Aspect
suffix
exclude?: string[]; | ||
} | ||
|
||
export abstract class TagBaseAspect implements IAspect { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd call this TagAspectBase
(that's the convention we have for abstract base classes)
child.apply(this); | ||
} | ||
} | ||
if (node instanceof Resource && (node as Resource).tagType !== TagType.NotTaggable) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use of instanceof
is not allowed since breaks when different version of Resource
are used. Use duck-typing to identify if the type adheres to the interface you are after (e.g. does it have a tagType
property?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will change, good information that I didn't know.
} | ||
|
||
public visit(node: Construct): void { | ||
if (this.visitedBy[node.uniqueId] === true) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that something we can add to the framework instead of the aspect itself?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was debating this, will move.
* | ||
* This object is rendered via a call to "renderProperties(this.properties)". | ||
*/ | ||
public readonly properties: any; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want this to be public.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, I'm going to have refactor renderProperties
to make a super call in the generated code. I'll propose something in the next refactor.
@@ -239,6 +242,47 @@ export class Resource extends Referenceable { | |||
} | |||
} | |||
|
|||
export enum TagType { | |||
Standard = "StandardTag", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Use single quotes
/** | ||
* Handles converting TagManager tags to AutoScalingGroupTags | ||
*/ | ||
export class AutoScalingGroupTagFormatter implements ITagFormatter { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we make all these formatters internal to the aspect code? No one else should touch those
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes they are already in the aspect this is just not cleaned up yet do to my rush on the draft.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, these are redundant now, right?
* TODO docs | ||
* can this be private? | ||
*/ | ||
public visit(): void { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about apply
and visit
. "apply" is not actually applying (it's "attaching") and "visit" is actually applying.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agree - even in Wikipedia they call apply Accept
and Visit
does the work. I'm open to name changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My naming suggestion here is semantic meaning from the perspective of the user, not mechanic meaning from the perspective of the implementor.
For the user, it looks like they are applying a tag to a construct, and the word apply is nice and generic enough that we can apply it (hah) to other types of aspects as well.
A mechanically correct alternative name would be attachAspect()
, but it doesn't read as nicely.
Compare:
// Customer first (what it does)
construct.apply(new cdk.Tag('key', 'value'));
// Implementor first (how it works)
construct.attachAspect(new cdk.TagAspect('key', 'value'));
At least, im(ns)ho :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I wouldn't call this particular function visit()
, I would call this something like invokeAspects()
.
And yes, it shouldn't be public, that will only serve to confuse people. It's not too bad, it should be invoked from App.run()
which is itself a construct so it can access protected
members.
* Aspects that can be applied to Constructs. | ||
*/ | ||
export interface IAspect { | ||
readonly type: string; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need this "type"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This type was included so we could filter on aspect types. For example, TagAspects check to see if there is another TagAspect that takes precedence. It's hard because parents and children can apply tags out of order and I had to rely on more than array order. There are other designs that could be done to make this work. For example, we could make the TagAspect collect all others TagAspects into an array and write a comparison function to always sort priority.
packages/@aws-cdk/aws-ec2/lib/vpc.ts
Outdated
// set first and let initial tags overwrite | ||
this.apply(new cdk.TagAspect(NAME_TAG, this.path)); | ||
const tags = props.tags || {}; | ||
for (const key of Object.keys(tags)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if it makes sense to support tags in the props
object anymore.
|
||
export interface TagProperties { | ||
// TODO docs | ||
propagate?: number; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still not sure that propagate
makes sense, and the number is "how deep"? All this machinery is purpose-built to allow selective tagging for VPCs, given that we know its internal construct layout, isn't it?
} | ||
this.visitedBy[node.uniqueId] = true; | ||
const propagate = this.propagate > 0 ? this.propagate - 1 : this.propagate; | ||
this.parent = this.parent || node; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you need this, it should also become part of the framework.
f.ex:
public visit(currentNode: Construct, applicationNode: Construct) {
fwiw I think it makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll review and incorporate if the other suggestions don't make this obsolete.
if (propagate !== 0) { | ||
for (const child of node.children) { | ||
this.propagate = propagate; | ||
child.apply(this); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, I think the framework should recursively apply.
} | ||
} | ||
|
||
export class TagAspect extends TagBaseAspect { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this is going to be contentious but I think I'd prefer this to not be suffixed with Aspect
.
construct.apply(new cdk.Tag('key', 'value'));
Reads better to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll agree this reads better and I originally I started here. For this draft I didn't want to rename the current cdk.cloudformation.Tag
class that we have. I did some looking and the specific class is mostly used in generated code. Would we prefer that we rename this to CfnTag
. I realize Cfn<resource>
now has special meaning and I think this is similar, but not identical. I also felt like the fact apply
(or another name we choose) will already set the mental context of the developer that this is an Aspect
and suffix is not necessary.
* TODO docs | ||
* can this be private? | ||
*/ | ||
public visit(): void { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My naming suggestion here is semantic meaning from the perspective of the user, not mechanic meaning from the perspective of the implementor.
For the user, it looks like they are applying a tag to a construct, and the word apply is nice and generic enough that we can apply it (hah) to other types of aspects as well.
A mechanically correct alternative name would be attachAspect()
, but it doesn't read as nicely.
Compare:
// Customer first (what it does)
construct.apply(new cdk.Tag('key', 'value'));
// Implementor first (how it works)
construct.attachAspect(new cdk.TagAspect('key', 'value'));
At least, im(ns)ho :)
* TODO docs | ||
* can this be private? | ||
*/ | ||
public visit(): void { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I wouldn't call this particular function visit()
, I would call this something like invokeAspects()
.
And yes, it shouldn't be public, that will only serve to confuse people. It's not too bad, it should be invoked from App.run()
which is itself a construct so it can access protected
members.
* specialized CloudFormation Resources (e.g. AutoScalingGroups, Serverless and | ||
* DAX) | ||
*/ | ||
export interface ITagFormatter { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deprecated, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes
if (propagate !== 0) { | ||
for (const child of node.children) { | ||
this.propagate = propagate; | ||
child.apply(this); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, in my head the Tags where physically attached only to the construct the user apply()
d them to, and only during visit()
ation would they also visit()
downward in the tree. Because... (:asterisk:, see below)
|
||
protected abstract applyTag(resource: Resource): void; | ||
|
||
private hasPrecedence(aspects: IAspect[]): boolean { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*️⃣, if we apply in order top-down in the tree, tags applied lower in the tree will naturally overwrite the tags applied higher in the tree, without having to write this function. Let's assume the following tree:
┌────────────────┐
│ │
│ Parent │ Tag(key, value1)
│ │
└────────────────┘
│
▼
┌────────────────┐
│ │
│ Child │ Tag(key, value2)
│ │
└────────────────┘
│
▼
┌────────────────┐
│ │
│TaggableResource│
│ │
└────────────────┘
This would resolve in the following way:
- The tag with
value1
would apply first. It would try to apply toParent
(doesn't do anything), then it would try to apply toChild
(doesn't do anything), then it would apply toTaggableResource
(setskey: value1
). - Next, the tag with
value2
would apply. It would try to apply toChild
(doesn't do anything), then it would apply toTaggableResource
. The value for tagkey
would be overwritten withvalue2
.
Presto, the more specific tag has taken precedence without any extra work on the aspect implementor's part.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you are correct. Let me work this in and see if my test cases still work.
@rix0rrr @eladb thanks for the comments. I hope to have them addressed tomorrow. Neither of you commented on include/exclude API. I've debated some name changes, but I think @rix0rrr is indicating we can kill |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feedback on comments inline.
} | ||
this.visitedBy[node.uniqueId] = true; | ||
const propagate = this.propagate > 0 ? this.propagate - 1 : this.propagate; | ||
this.parent = this.parent || node; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll review and incorporate if the other suggestions don't make this obsolete.
I would prefer a more generic selector mechanism that also works for other aspects. However I recognize that we don't have a great solution for this right now, and at least it doesn't harm to have a mechanism inside the Tag aspect system as well; we can always refactor later. So I'm okay with keeping it in for now. |
That may get you into trouble for unit tests, btw. But in those cases we can always do the pragmatic thing of casting to |
b05e492
to
7eda0c2
Compare
* The base Aspect class is added and is an idempotent propagating visitor, that recurses during the visit action * The first aspects are for tags * TagManager has been re-implemented to only handle setting removing and formatting tags * Tags can be included or excluded based on Resource Type, this replaces the the need for all previous tag properties * VPC, Security Group, KMS Keys, and AutoScaling are updated to use tags * Tag Aspects overwrite tags passed into the L1 during initialization
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
test.deepEqual(map.tags.renderTags(), {foo: 'bar'}); | ||
test.done(); | ||
}, | ||
'Aspects are mutually exclusive with tags created by L1 Constructor'(test: Test) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an important behavior to note. I could make this more complicated and solve merges. However, I think it's easy to move to the new Tag
and keeps the code simple.
* The base Aspect class is added and is an idempotent propagating visitor, that recurses during the visit action * The first aspects are for tags * TagManager has been re-implemented to only handle setting removing and formatting tags * Tags can be included or excluded based on Resource Type, this replaces the the need for all previous tag properties * VPC, Security Group, KMS Keys, and AutoScaling are updated to use tags * Tag Aspects overwrite tags passed into the L1 during initialization
0478e6b
to
0109956
Compare
Ok I'm at test passing, unit test complete, and jsdocs complete. I added a I didn't create any documentation for Aspect README yet. Where should I put that and should I write it? |
@@ -0,0 +1,7 @@ | |||
import cdk = require('@aws-cdk/cdk'); | |||
|
|||
export class TestStack extends cdk.Stack { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Is this a good idea to put in a reusable place?
- I started to find all the places we should use this, but I missed a few and now I'm questioning if this is too much to include here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this class is no longer needed but the test clean up seems like it was a good idea?
|
||
export class TestStack extends cdk.Stack { | ||
public testInvokeAspects() { | ||
this.invokeAspects(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we move invokeAspects behind the all new “prepareTree”?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I think that makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for clarity, I am doing:
- remove
invokeAspects()
fromapp.ts
- putting
invokeAspects()
inside ofprepareTree()
- removing the
TestStack
becauseprepareTree()
is public and called duringexpect
Is there any concern about the conditional wrapped around the block that calls prepareTree()
?
if (!skipValidation) { // any concern here - I haven't found any but just double checking
// Do a prepare-and-validate run over the given stack
stack.node.prepareTree();
const errors = stack.node.validateTree();
if (errors.length > 0) {
throw new Error(`Stack validation failed:\n${errors.map(e => `${e.message} at: ${e.source.node.scope}`).join('\n')}`);
}
}
// this will be removed
if (isTestStack(stack)) {
stack.testInvokeAspects();
}
// end of block to be removed
A simple thumbs up to verify is fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lgtm; Curious why prepare tree is skipped if validation is skipped but that’s for a different issue (raise one?).
* Node will only be visited once. | ||
*/ | ||
public visit(construct: IConstruct): void { | ||
if (this.visitedBy[construct.node.uniqueId] === true) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’d rename this to “visitTree” and “visitAction” to “visit”
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A little clarification. Does this also mean change:
/**
* Represents an Aspect
*/
export interface IAspect {
/**
* The type of Aspect
*/
readonly type: string;
/**
* All aspects can visit by IConstructs
*/
visit(node: IConstruct): void;
}
to
/**
* Represents an Aspect
*/
export interface IAspect {
/**
* The type of Aspect
*/
readonly type: string;
/**
* All aspects can visit the construct tree starting from this construct
*/
visitTree(node: IConstruct): void;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn’t say so. I think aspects are only expected to deal with their own visit. The tree visitation is handled by the framework, no?
Please update the PR title + description to match the commit description you wish us to merge eventually |
EC2 task definitions can now be used as CloudWatch event targets. ALSO IN THIS COMMIT * Improve hash calculation of Docker images. * Add `grantPassRole()` method to iam.Role Fixes aws#1370.
Add source map support to the TypeScript app entry point template. This won't retroactively fix existing CDK apps, but it will ensure that apps generated in the future via `cdk init --app` will have it enabled. Fixes aws#1579
The property was modeled in the wrong place, rendering it unusable. Moved it to the correct location instead, and added a validation test. Fixes aws#1608
Constructs can now take a dependency on any other construct. Before, only `Resource`s could take dependencies, and they would depend on `IDependable` which had to be implemented explicitly. In this change we generalize the concept of dependencies from construct trees to other construct trees; all constructs now take dependencies and also implement `IDependable`. The semantics are that any resource in the depending tree will depend on all resources in the depended tree. Dependencies are cross-stack aware If you take a dependency on a construct in another stack, the dependency does not get rendered in the template, but is instead added as a dependency between stacks. Fixes aws#1568, fixes aws#95. BREAKING CHANGE: `resource.addDependency()` has been moved onto `ConstructNode`. You now write `resource.node.addDependency()`. VPC's `internetDependency` has been moved to the subnets as `internetConnectivityEstablished`. Target Group's `loadBalancerAssociationDependencies` has been renamed to `loadBalancerAttached`.
Adds `LambdaApplication` and `LambdaDeploymentGroup` such that an `Alias` to a Lambda `Function` can be deployed via CodeDeploy; supports pre/post hook functions, traffic-shifting and alarm monitoring and rollback.
…#1664) Adds option to enable all block public access settings on the bucket (`blockPublicAccess`). Also throws an error when trying to use grantPublicAccess if blockPublicAccess is set to true. https://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-block-public-access.html
A generalized aspect framework is added. Aspects can be used to affect the construct tree without modifying the class hierarchy. This framework is designed to be generic for future use cases. Tagging is the first implementation. Tagging has been reimplemented to leverage the aspect framework and simplify the original tag design. Tag Manager still exists, but is no longer intended for use by L2 construct authors. There are two new classes `cdk.Tag` and `cdk.RemoveTag`. As new resources are added tag support will be automatic as long as they implement one of the existing tag specifications. All L2 constructs have removed the TagManager. Code generation has been modified to add tag support to any CloudFormation Resource with a matching specification, by creating a Tag Manager for that resource as a `tags` attribute. The schema code now includes the ability to detect 3 forms of tags which include the current CloudFormation Resources. BREAKING CHANGE: if you are using TagManager the API for this object has completely changed. You should no longer use TagManager directly, but instead replace this with Tag Aspects. `cdk.Tag` has been renamed to `cdk.CfnTag` to enable `cdk.Tag` to be the Tag Aspect. Fixes aws#1136.
@eladb - I think tests are finally passing. Is there anything else? |
A generalized aspect framework is added. Aspects can be used to affect the construct tree without modifying the class hierarchy. This framework is designed to be generic for future use cases. Tagging is the first implementation.
Tagging has been reimplemented to leverage the aspect framework and simplify the original tag design. Tag Manager still exists, but is no longer intended for use by L2 construct authors. There are two new classes
cdk.Tag
andcdk.RemoveTag
. As new resources are added tag support will be automatic as long as they implement one of the existing tag specifications. All L2 constructs have removed the TagManager.Code generation has been modified to add tag support to any CloudFormation Resource with a matching specification, by creating a Tag Manager for that resource as a
tags
attribute. The schema code now includes the ability to detect 3 forms of tags which include the current CloudFormation Resources.BREAKING CHANGE: if you are using TagManager the API for this object has completely changed. You should no longer use TagManager directly, but instead replace this with Tag Aspects.
cdk.Tag
has been renamed tocdk.CfnTag
to enablecdk.Tag
to be the Tag Aspect.Fixes #1136
Fixes #1497
Pull Request Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.