Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ec2/ecs): cacheInContext properties for machine images #16021

Merged
merged 7 commits into from
Sep 9, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 149 additions & 5 deletions packages/@aws-cdk/aws-ec2/lib/machine-image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,27 @@ export abstract class MachineImage {
* @param parameterName The name of SSM parameter containing the AMi id
* @param os The operating system type of the AMI
* @param userData optional user data for the given image
* @deprecated Use `MachineImage.fromSsmParameter()` instead
*/
public static fromSSMParameter(parameterName: string, os: OperatingSystemType, userData?: UserData): IMachineImage {
return new GenericSSMParameterImage(parameterName, os, userData);
}

/**
* An image specified in SSM parameter store
*
* By default, the SSM parameter is refreshed at every deployment,
* causing your instances to be replaced whenever a new version of the AMI
* is released.
*
* Pass `{ cachedInContext: true }` to keep the AMI ID stable. If you do, you
* will have to remember to periodically invalidate the context to refresh
* to the newest AMI ID.
*/
public static fromSsmParameter(parameterName: string, options?: SsmParameterImageOptions): IMachineImage {
return new GenericSsmParameterImage(parameterName, options);
}

/**
* Look up a shared Machine Image using DescribeImages
*
Expand All @@ -96,6 +112,8 @@ export abstract class MachineImage {
* will be used on future runs. To refresh the AMI lookup, you will have to
* evict the value from the cache using the `cdk context` command. See
* https://docs.aws.amazon.com/cdk/latest/guide/context.html for more information.
*
* This function can not be used in environment-agnostic stacks.
*/
public static lookup(props: LookupMachineImageProps): IMachineImage {
return new LookupMachineImage(props);
Expand Down Expand Up @@ -131,10 +149,17 @@ export interface MachineImageConfig {
* on the instance if you are using this image.
*
* The AMI ID is selected using the values published to the SSM parameter store.
*
* @deprecated Use `MachineImage.fromSsmParameter()` instead
BenChaimberg marked this conversation as resolved.
Show resolved Hide resolved
*/
export class GenericSSMParameterImage implements IMachineImage {
/**
* Name of the SSM parameter we're looking up
*/
public readonly parameterName: string;

constructor(private readonly parameterName: string, private readonly os: OperatingSystemType, private readonly userData?: UserData) {
constructor(parameterName: string, private readonly os: OperatingSystemType, private readonly userData?: UserData) {
this.parameterName = parameterName;
}

/**
Expand All @@ -150,6 +175,75 @@ export class GenericSSMParameterImage implements IMachineImage {
}
}

/**
* Properties for GenericSsmParameterImage
*/
export interface SsmParameterImageOptions {
/**
* Operating system
*
* @default OperatingSystemType.LINUX
*/
readonly os?: OperatingSystemType;

/**
* Custom UserData
*
* @default - UserData appropriate for the OS
*/
readonly userData?: UserData;

/**
* Whether the AMI ID is cached to be stable between deployments
*
* By default, the newest image is used on each deployment. This will cause
* instances to be replaced whenever a new version is released, and may cause
* downtime if there aren't enough running instances in the AutoScalingGroup
* to reschedule the tasks on.
*
* If set to true, the AMI ID will be cached in `cdk.context.json` and the
* same value will be used on future runs. Your instances will not be replaced
* but your AMI version will grow old over time. To refresh the AMI lookup,
* you will have to evict the value from the cache using the `cdk context`
* command. See https://docs.aws.amazon.com/cdk/latest/guide/context.html for
* more information.
*
* Can not be set to `true` in environment-agnostic stacks.
*
* @default false
*/
readonly cachedInContext?: boolean;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just "cache" would be fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want people thinking about "context" though...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, this feels like an implementation detail that doesn't need to be surfaced in the API name. All the customer cares is that the AMI does not change automatically without cache eviction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you in general. What concerns me is that users might not know where to look for said cache eviction--which is why I want that word in there.

}

/**
* Select the image based on a given SSM parameter
*
* This Machine Image automatically updates to the latest version on every
* deployment. Be aware this will cause your instances to be replaced when a
* new version of the image becomes available. Do not store stateful information
* on the instance if you are using this image.
*
* The AMI ID is selected using the values published to the SSM parameter store.
*/
class GenericSsmParameterImage implements IMachineImage {
constructor(private readonly parameterName: string, private readonly props: SsmParameterImageOptions = {}) {
}

/**
* Return the image to use in the given context
*/
public getImage(scope: Construct): MachineImageConfig {
const imageId = lookupImage(scope, this.props.cachedInContext, this.parameterName);

const osType = this.props.os ?? OperatingSystemType.LINUX;
return {
imageId,
osType,
userData: this.props.userData ?? (osType === OperatingSystemType.WINDOWS ? UserData.forWindows() : UserData.forLinux()),
};
}
}

/**
* Configuration options for WindowsImage
*/
Expand Down Expand Up @@ -240,6 +334,27 @@ export interface AmazonLinuxImageProps {
* @default X86_64
*/
readonly cpuType?: AmazonLinuxCpuType;

/**
* Whether the AMI ID is cached to be stable between deployments
*
* By default, the newest image is used on each deployment. This will cause
* instances to be replaced whenever a new version is released, and may cause
* downtime if there aren't enough running instances in the AutoScalingGroup
* to reschedule the tasks on.
*
* If set to true, the AMI ID will be cached in `cdk.context.json` and the
* same value will be used on future runs. Your instances will not be replaced
* but your AMI version will grow old over time. To refresh the AMI lookup,
* you will have to evict the value from the cache using the `cdk context`
* command. See https://docs.aws.amazon.com/cdk/latest/guide/context.html for
* more information.
*
* Can not be set to `true` in environment-agnostic stacks.
*
* @default false
*/
readonly cachedInContext?: boolean;
}

/**
Expand All @@ -253,8 +368,10 @@ export interface AmazonLinuxImageProps {
* The AMI ID is selected using the values published to the SSM parameter store.
*/
export class AmazonLinuxImage extends GenericSSMParameterImage {

constructor(props: AmazonLinuxImageProps = {}) {
/**
* Return the SSM parameter name that will contain the Amazon Linux image with the given attributes
*/
public static ssmParameterName(props: AmazonLinuxImageProps = {}) {
const generation = (props && props.generation) || AmazonLinuxGeneration.AMAZON_LINUX;
const edition = (props && props.edition) || AmazonLinuxEdition.STANDARD;
const virtualization = (props && props.virtualization) || AmazonLinuxVirt.HVM;
Expand All @@ -269,8 +386,29 @@ export class AmazonLinuxImage extends GenericSSMParameterImage {
storage,
].filter(x => x !== undefined); // Get rid of undefineds

const parameterName = '/aws/service/ami-amazon-linux-latest/' + parts.join('-');
super(parameterName, OperatingSystemType.LINUX, props.userData);
return '/aws/service/ami-amazon-linux-latest/' + parts.join('-');
}

private readonly cachedInContext: boolean;

constructor(private readonly props: AmazonLinuxImageProps = {}) {
super(AmazonLinuxImage.ssmParameterName(props), OperatingSystemType.LINUX, props.userData);

this.cachedInContext = props.cachedInContext ?? false;
}

/**
* Return the image to use in the given context
*/
public getImage(scope: Construct): MachineImageConfig {
BenChaimberg marked this conversation as resolved.
Show resolved Hide resolved
const imageId = lookupImage(scope, this.cachedInContext, this.parameterName);

const osType = OperatingSystemType.LINUX;
return {
imageId,
osType,
userData: this.props.userData ?? UserData.forLinux(),
};
}
}

Expand Down Expand Up @@ -536,3 +674,9 @@ export interface LookupMachineImageProps {
*/
readonly userData?: UserData;
}

function lookupImage(scope: Construct, cachedInContext: boolean | undefined, parameterName: string) {
return cachedInContext
? ssm.StringParameter.valueFromLookup(scope, parameterName)
: ssm.StringParameter.valueForTypedStringParameter(scope, parameterName, ssm.ParameterType.AWS_EC2_IMAGE_ID);
}
19 changes: 19 additions & 0 deletions packages/@aws-cdk/aws-ec2/test/machine-image.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,25 @@ test('LookupMachineImage creates correct type of UserData', () => {
expect(isLinuxUserData(linuxDetails.userData)).toBeTruthy();
});

test('cached lookups of Amazon Linux', () => {
// WHEN
const ami = ec2.MachineImage.latestAmazonLinux({ cachedInContext: true }).getImage(stack).imageId;

// THEN
expect(ami).toEqual('dummy-value-for-/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2');
expect(app.synth().manifest.missing).toEqual([
{
key: 'ssm:account=1234:parameterName=/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2:region=testregion',
props: {
account: '1234',
region: 'testregion',
parameterName: '/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2',
},
provider: 'ssm',
},
]);
});

function isWindowsUserData(ud: ec2.UserData) {
return ud.render().indexOf('powershell') > -1;
}
Expand Down
17 changes: 17 additions & 0 deletions packages/@aws-cdk/aws-ecs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,23 @@ cluster.addAutoScalingGroup(autoScalingGroup);

If you omit the property `vpc`, the construct will create a new VPC with two AZs.

By default, all machine images will auto-update to the latest version
on each deployment, causing a replacement of the instances in your AutoScalingGroup
if the AMI has been updated since the last deployment.

If task draining is enabled, ECS will transparently reschedule tasks on to the new
instances before terminating your old instances. If you have disabled task draining,
BenChaimberg marked this conversation as resolved.
Show resolved Hide resolved
the tasks will be terminated along with the instance. To prevent that, you
can pick a non-updating AMI by passing `cacheInContext: true`, but be sure
to periodically update to the latest AMI manually by using the [CDK CLI
context management commands](https://docs.aws.amazon.com/cdk/latest/guide/context.html):

```ts
const autoScalingGroup = new autoscaling.AutoScalingGroup(this, 'ASG', {
// ...
machineImage: EcsOptimizedImage.amazonLinux({ cacheInContext: true }),
});
```

### Bottlerocket

Expand Down
Loading