Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ecs): enable alarm-based rollbacks #25840

Merged
merged 46 commits into from
Jul 5, 2023
Merged

Conversation

bvtujo
Copy link
Contributor

@bvtujo bvtujo commented Jun 2, 2023

This PR enables the new Deployment Alarms feature in ECS for Service L2s blog post.

This PR contains changes after the first round of revision by the CDK team.

Continuation of #25346

Why are so many integration tests impacted by this change? There are 2 reasons:

  1. This PR changes the ECS L2s to set the default configuration for the CfnService.deploymentConfiguration.alarms property to:
alarmNames: [],
rollback: false,
enable: false,

This is necessary, because adding deployment alarms, deploying your CFN stack, then removing the deployment alarms from the CFN template, and deploying again WILL NOT remove the deployment alarms from the service. To remove previously configured deployment alarms, you must explicitly use the configuration shown above. Making this update will cause no interruption to existing ECS services, and does not trigger any update to the service itself during the CFN update.

The ECS UpdateService API is stateful, meaning that if a field is not present in the CloudFormation object, it will be ignored in the update. This was originally implemented due to the problems with desiredCount resetting to bad values after autoscaling changed it, but requires us to set an explicit disableDeploymentAlarms() method to cause the service update to behave correctly.

  1. Most ECS integ tests have not been executed in a long time. So, many of the changes are bring the snapshots up to date with the latest integ test format, or lambda layers, etc.

Detailed list of changes:

  • entire alarm based rollback task into single commit
  • resolving merge conflicts
  • integ files modules updated to resolve merge conflicts
  • regarding merge conflicts issue
  • integ files import changes for merge conflicts
  • adding this change for PR not to abondended
  • chore: refactor createAlarm & remove unnecessary unit tests
  • chore: fix api but failing unit tests
  • chore: fix validation
  • chore: readme changes
  • chore: fix integ test, remove unnecessary ec2 test
  • chore: refactor enableAlarms for clarity
  • test: fix integ test, finally
  • docs: update docstrings for new props and methods

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Naumel and others added 30 commits March 28, 2023 21:54
Closes aws#24243.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Copy link
Contributor

@madeline-k madeline-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @bvtujo, thanks for your patience on this one. It was a bit of a challenge.

All of my feedback I have implemented in another branch, which you can merge into your branch if you are aligned with it. Checkout the commit here: 8017b6f?diff=split

packages/aws-cdk-lib/aws-ecs/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
@mergify mergify bot dismissed madeline-k’s stale review July 3, 2023 12:37

Pull request has been modified.

Copy link
Contributor

@madeline-k madeline-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thanks for incorporating all of my feedback. I have a few tiny comments remaining on the documentation.

packages/aws-cdk-lib/aws-ecs/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/README.md Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts Outdated Show resolved Hide resolved
@mergify mergify bot dismissed madeline-k’s stale review July 5, 2023 15:10

Pull request has been modified.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 644c468
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

@madeline-k madeline-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jul 5, 2023
@mergify
Copy link
Contributor

mergify bot commented Jul 5, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 1a44f69 into aws:main Jul 5, 2023
mergify bot pushed a commit that referenced this pull request Jul 12, 2023
… CODE_DEPLOY and EXTERNAL deployment controller (#26317)

From #25840, ECS L2 construct sets the default configuration for the `CfnService.deploymentConfiguration.alarms` property to:
```
alarmNames: [],
rollback: false,
enable: false,
```

However, alarm based rollback feature is only supported for ECS services that use the rolling update (ECS) deployment controller.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deployment-alarm-failure.html

Due to this limitation, when deploymentController is set to CODE_DEPLOY or EXTERNAL, creation for the service will fail by conflict with `deploymentConfiguration.alarms` property.

This PR solves the issue by skipping to set default configuration for the `CfnService.deploymentConfiguration.alarms` property for CODE_DEPLOY and EXTERNAL deployment controller.

Closes #26307

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
bmoffatt pushed a commit to bmoffatt/aws-cdk that referenced this pull request Jul 29, 2023
… CODE_DEPLOY and EXTERNAL deployment controller (aws#26317)

From aws#25840, ECS L2 construct sets the default configuration for the `CfnService.deploymentConfiguration.alarms` property to:
```
alarmNames: [],
rollback: false,
enable: false,
```

However, alarm based rollback feature is only supported for ECS services that use the rolling update (ECS) deployment controller.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deployment-alarm-failure.html

Due to this limitation, when deploymentController is set to CODE_DEPLOY or EXTERNAL, creation for the service will fail by conflict with `deploymentConfiguration.alarms` property.

This PR solves the issue by skipping to set default configuration for the `CfnService.deploymentConfiguration.alarms` property for CODE_DEPLOY and EXTERNAL deployment controller.

Closes aws#26307

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
mergify bot pushed a commit that referenced this pull request May 15, 2024
…ttings (#30217)

### Issue # (if applicable)

Internal ticket tracking V1142791950

### Reason for this change

Originally in this PR #25840, we added default deployment alarm settings to fix an issue where adding deployment alarms, deploying your CFN stack, then removing the deployment alarms from the CFN template, and deploying again WILL NOT remove the deployment alarms from the service.

ECS now already supports default deployment alarm settings. We will remove the default setting of deploymentAlarms. Reason for removing this default behaviour is for region build where the deployment alarm service may not be available in new regions but is set by default by CDK.

### Description of changes

Remove default deployment alarm.

### Description of how you validated changes

All new tests and integration tests pass.

### Checklist
- [ ] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
atanaspam pushed a commit to atanaspam/aws-cdk that referenced this pull request Jun 3, 2024
…ttings (aws#30217)

### Issue # (if applicable)

Internal ticket tracking V1142791950

### Reason for this change

Originally in this PR aws#25840, we added default deployment alarm settings to fix an issue where adding deployment alarms, deploying your CFN stack, then removing the deployment alarms from the CFN template, and deploying again WILL NOT remove the deployment alarms from the service.

ECS now already supports default deployment alarm settings. We will remove the default setting of deploymentAlarms. Reason for removing this default behaviour is for region build where the deployment alarm service may not be available in new regions but is set by default by CDK.

### Description of changes

Remove default deployment alarm.

### Description of how you validated changes

All new tests and integration tests pass.

### Checklist
- [ ] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants