feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and reduce aws-cdk-lib size #22677

kaizencc · 2022-10-27T20:42:29Z

This PR reduces the size of aws-cdk-lib by 25.6 MB, by extracting the lambda-layer-kubectl/lib/layer.zip file to a separate package that aws-cdk-lib will directly depend on.

This change also reduce the complexity of the build in the aws-cdk repo because the process of building the layer.zip inside of a Docker container has also been extracted to the separate package. See https://github.com/cdklabs/awscdk-asset-kubectl/tree/kubectl-v20/main

All Submissions:

Have you followed the guidelines in our Contributing guide?

Adding new Unconventional Dependencies:

This PR adds new unconventional dependencies following the process described here

New Features

Have you added the new feature to an integration test?
- Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

…emove zip files

gitpod-io · 2022-10-27T20:42:39Z

packages/@aws-cdk/lambda-layer-kubectl/lib/kubectl-layer.ts

madeline-k

Looks good to me!

packages/@aws-cdk/lambda-layer-kubectl/lib/kubectl-layer.ts

…o conroy/removezip

TheRealAmazonKendra · 2022-11-02T17:20:30Z

It doesn't seem to be showing the actual build logs here properly, but it looks like you need to add the yarn.lock changes:

Error: ERROR! Dependency @aws-cdk/asset-kubectl-v20@^2.1.0 from @aws-cdk/lambda-layer-kubectl not present in yarn.lock. Please run 'yarn install' and try again!
--
493 | at errorIfNotInYarnLock (/codebuild/output/src846888144/src/github.com/aws/aws-cdk/scripts/check-yarn-lock.js:45:13)
494 | at /codebuild/output/src846888144/src/github.com/aws/aws-cdk/scripts/check-yarn-lock.js:51:77
495 | at Array.forEach (<anonymous>)
496 | at /codebuild/output/src846888144/src/github.com/aws/aws-cdk/scripts/check-yarn-lock.js:51:42
497 | at Array.forEach (<anonymous>)
498 | at main (/codebuild/output/src846888144/src/github.com/aws/aws-cdk/scripts/check-yarn-lock.js:49:12)
499 |  
500 | [Container] 2022/11/02 16:31:14 Command did not exit successfully /bin/bash ./build.sh exit status 1
501 | [Container] 2022/11/02 16:31:14 Phase complete: BUILD State: FAILED
502 | [Container] 2022/11/02 16:31:14 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: /bin/bash ./build.sh. Reason: exit status 1
503 | [Container] 2022/11/02 16:31:14 Entering phase POST_BUILD
504 | [Container] 2022/11/02 16:31:14 Phase complete: POST_BUILD State: SUCCEEDED
505 | [Container] 2022/11/02 16:31:14 Phase context status code:  Message:

madeline-k

Wait, shouldn't this also modify aws-cdk-lib's package.json?

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

✅ Updated pull request passes all PRLinter validations. Dissmissing previous PRLinter review.

TheRealAmazonKendra · 2022-11-03T02:29:37Z

Pretty sure because of #22716 you'll just need to rebase and rerun yarn integ --update-on-failed

kaizencc · 2022-11-04T20:32:35Z

packages/@aws-cdk/aws-stepfunctions-tasks/test/emrcontainers/integ.start-job-run.ts

+  cdkCommandOptions: {
+    deploy: {
+      args: {
+        rollback: true,
+      },
+    },
+  },


3 tests were in need of update in stepfunctions-tasks, because they use a CDK construct that depends on lambda-layer-kubectl, and we updated the asset hash for the layer. cdk deploy has a normal default of rollback: true, but our test runner defaults to rollback: false. According to @corymhall, this is because of the assertion stack; we want them all to run even if some of them fail. There are no assertions here, so I feel safe turning on rollback: true.

However, calling this out: with rollback: false, the update workflow fails with Replacement type updates not supported on stack with disable-rollback on the KubectlLayer. It looks like on update, the different asset hash requires replacement of the lambda layer. And any sort of replacement is not allowed when rollback: false, a rule that comes from cloudformation and not the resource itself. I think we should be okay with this, but I definitely want a second opinion. @madeline-k, @rix0rrr, what do you think?

kaizencc · 2022-11-07T21:03:50Z

@Mergifyio refresh

mergify · 2022-11-07T21:04:15Z

refresh

✅ Pull request refreshed

addressed

aws-cdk-automation · 2022-11-07T22:32:18Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: b959b97
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2022-11-07T22:32:29Z

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

chore(lambda-layer-kubectl): depend on awscdk-asset-kubectl-v20 and r…

bdb3598

…emove zip files

aws-cdk-automation requested a review from a team October 27, 2022 20:42

github-actions bot added the p2 label Oct 27, 2022

mergify bot added the contribution/core This is a PR that came from AWS. label Oct 27, 2022

kaizencc added 2 commits October 31, 2022 17:45

DOCKER_DIR to LAMBDA_SOURCE_DIR

c433534

change hashing

f93e226

kaizencc commented Oct 31, 2022

View reviewed changes

packages/@aws-cdk/lambda-layer-kubectl/lib/kubectl-layer.ts Outdated Show resolved Hide resolved

rix0rrr approved these changes Nov 1, 2022

View reviewed changes

madeline-k reviewed Nov 1, 2022

View reviewed changes

packages/@aws-cdk/lambda-layer-kubectl/lib/kubectl-layer.ts Outdated Show resolved Hide resolved

kaizencc marked this pull request as ready for review November 2, 2022 15:49

kaizencc and others added 5 commits November 2, 2022 11:50

use new kubectl-v20 library

9f45481

Merge branch 'main' into conroy/removezip

d2c802d

update pkglint rules to exclude kubectl-v20

e6c4448

Merge branch 'conroy/removezip' of https://github.com/aws/aws-cdk int…

6e1eeb1

…o conroy/removezip

Empty-Commit

1c5c9c8

kaizencc and others added 2 commits November 2, 2022 15:12

yarn.lock

5d2f6c4

yarn.lock & integration test

e1c0eef

madeline-k previously requested changes Nov 2, 2022

View reviewed changes

madeline-k changed the title ~~chore(lambda-layer-kubectl): depend on awscdk-asset-kubectl-v20 and remove zip files~~ chore(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and remove zip file Nov 2, 2022

madeline-k changed the title ~~chore(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and remove zip file~~ feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and remove zip file Nov 2, 2022

madeline-k changed the title ~~feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and remove zip file~~ feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and reduce aws-cdk-lib size Nov 2, 2022

aws-cdk-automation previously requested changes Nov 2, 2022

View reviewed changes

madeline-k added the pr-linter/exempt-readme The PR linter will not require README changes label Nov 2, 2022

kaizencc and others added 2 commits November 3, 2022 09:57

Merge branch 'main' into conroy/removezip

10be8cc

integ test

341cd56

kaizencc added the pr/do-not-merge This PR should not be merged at this time. label Nov 3, 2022

kaizencc added 3 commits November 3, 2022 10:43

remove old testing folder;

1c8b9f1

add dep in package.json

f600e12

stepfunction test updates

861a0f9

kaizencc commented Nov 4, 2022

View reviewed changes

kaizencc removed the pr/do-not-merge This PR should not be merged at this time. label Nov 7, 2022

Merge branch 'main' into conroy/removezip

94a5574

integ test snapshot

b959b97

mergify bot merged commit 6c606d0 into main Nov 7, 2022

mergify bot deleted the conroy/removezip branch November 7, 2022 22:32

AlyIbrahim mentioned this pull request Jan 30, 2023

aws_eks: error retrieving RESTMappings to prune #23376

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and reduce aws-cdk-lib size #22677

feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and reduce aws-cdk-lib size #22677

kaizencc commented Oct 27, 2022 •

edited by madeline-k

Loading

gitpod-io bot commented Oct 27, 2022

madeline-k left a comment

TheRealAmazonKendra commented Nov 2, 2022

madeline-k left a comment

aws-cdk-automation left a comment

TheRealAmazonKendra commented Nov 3, 2022 •

edited

Loading

kaizencc Nov 4, 2022

kaizencc commented Nov 7, 2022

mergify bot commented Nov 7, 2022

aws-cdk-automation commented Nov 7, 2022

mergify bot commented Nov 7, 2022

feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and reduce aws-cdk-lib size #22677

feat(lambda-layer-kubectl): depend on @awscdk/asset-kubectl-v20 and reduce aws-cdk-lib size #22677

Conversation

kaizencc commented Oct 27, 2022 • edited by madeline-k Loading

All Submissions:

Adding new Unconventional Dependencies:

New Features

gitpod-io bot commented Oct 27, 2022

madeline-k left a comment

Choose a reason for hiding this comment

TheRealAmazonKendra commented Nov 2, 2022

madeline-k left a comment

Choose a reason for hiding this comment

aws-cdk-automation left a comment

Choose a reason for hiding this comment

TheRealAmazonKendra commented Nov 3, 2022 • edited Loading

kaizencc Nov 4, 2022

Choose a reason for hiding this comment

kaizencc commented Nov 7, 2022

mergify bot commented Nov 7, 2022

✅ Pull request refreshed

aws-cdk-automation commented Nov 7, 2022

AWS CodeBuild CI Report

mergify bot commented Nov 7, 2022

kaizencc commented Oct 27, 2022 •

edited by madeline-k

Loading

TheRealAmazonKendra commented Nov 3, 2022 •

edited

Loading