awscdk-kubectl-go: KubectlLayer modules out of date, broken by Python 3.7 runtime removal #29123

jaredhancock31 · 2024-02-15T20:14:41Z

Describe the bug

Python 3.7 runtime support was recently dropped.

For those who use aws-cdk-go, we have to import separate KubectlLayers after K8s 1.21. But these layers are extremely outdated, and now broken.

1.25 example: https://github.com/cdklabs/awscdk-kubectl-go/blob/kubectl.25/kubectlv25/go.mod#L7

they all use aws-cdk 2.28 and are still on Go 1.18.

We use aws-cdk-go and these layer libraries in production, so this prevents any stack updates until we hack a workaround or new tags are pushed in the cdklabs repo.

Expected Behavior

EKS cluster stack should successfully be created

Current Behavior

stack fails with errors like

Embedded stack XXXX-awscdkawseksKubectlProviderNestedStackawscdkawseksKubectlPro-XXXXXX was not successfully created: The following resource(s) failed to create: [Handler886CB40B]

"The runtime parameter of python3.7 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (python3.12) while creating or updating functions. "

when checking the rendered cloudformation it looks like this:

   "Role":"arn:aws:iam::REDACTED-HandlerServiceRoleFCDC14A-REDACTED",
   "MemorySize":"1024",
   "Runtime":"python3.7",
   "Description":"onEvent handler for EKS kubectl resource provider",
   "Timeout":"900",
   "Handler":"index.handler",
   "Code":{
      "S3Bucket":"REDACTED",
      "S3Key":"REDACTED.zip"
   },
   "Layers":[
      "arn:aws:lambda:REDACTED:layer:AwsCliLayerF44AAF94:58",
      "arn:aws:lambda:REDACTED:layer:KubectlV25Layer182E5352:29"
   ]

Reproduction Steps

create an awseks.Cluster construct with kubectl layer like so:

props := &awseks.ClusterProps{
    KubectlLayer = getKubectlLayer(stack, version)
}

cluster := awseks.NewCluster(stack, jsii.String("EKSCluster"), props)

import (
	layer22 "github.com/cdklabs/awscdk-kubectl-go/kubectlv22/v2"
	layer23 "github.com/cdklabs/awscdk-kubectl-go/kubectlv23/v2"
	layer24 "github.com/cdklabs/awscdk-kubectl-go/kubectlv24/v2"
	layer25 "github.com/cdklabs/awscdk-kubectl-go/kubectlv25/v2"
)

func getKubectlLayer(stack awscdk.Stack, k8sVersion string) awslambda.LayerVersion {
	switch k8sVersion {
	case "1.22":
		return layer22.NewKubectlV22Layer(stack, jsii.String("KubectlV22Layer"))
	case "1.23":
		return layer23.NewKubectlV23Layer(stack, jsii.String("KubectlV23Layer"))
	case "1.24":
		return layer24.NewKubectlV24Layer(stack, jsii.String("KubectlV24Layer"))
	case "1.25":
		return layer25.NewKubectlV25Layer(stack, jsii.String("KubectlV25Layer"))
	}
	return nil
}

Possible Solution

update libs at https://github.com/cdklabs/awscdk-kubectl-go

Additional Information/Context

No response

CDK CLI Version

2.80.0

Framework Version

No response

Node.js Version

18

OS

Mac/Linux

Language

Go

Language Version

1.20

Other information

No response

The text was updated successfully, but these errors were encountered:

pahud · 2024-02-16T16:04:19Z

Thank you for the report. We'll look into this issue.

moelasmar · 2024-02-21T00:09:59Z

Hey @jaredhancock31 for raising this issue,

Can you elaborate more about your issue, as I am able to create a Lambda function with Python 3.9 using aws-cdk-go v2.28.0, and add a kubectl layer to it using awscdk-kubectl-go/kubectlv25 v2.0.4 and run cdk synth and cdk deploy successfully. What I am missing is what is the problem from your side to upgrade your lambda function runtime to use Python 3.9

see the following snippets:

\\ go.mod

module app_kubectl_go

go 1.18

require (
	github.com/aws/aws-cdk-go/awscdk/v2 v2.28.0
	github.com/aws/constructs-go/constructs/v10 v10.1.33
	github.com/aws/jsii-runtime-go v1.76.0
	github.com/cdklabs/awscdk-kubectl-go/kubectlv25/v2 v2.0.4
)

// app_kubectl_go.go
....
func NewAppKubectlGoStack(scope constructs.Construct, id string, props *AppKubectlGoStackProps) awscdk.Stack {
	var sprops awscdk.StackProps
	if props != nil {
		sprops = props.StackProps
	}
	stack := awscdk.NewStack(scope, &id, &sprops)
	baseDir, err := os.Getwd()

	if err != nil {
		fmt.Println("Failed to get current working directory", err)
		return nil
	}

	fn := lambda.NewFunction(stack, jsii.String("MyFunction"), &lambda.FunctionProps{
		Runtime: lambda.Runtime_PYTHON_3_9(),
		Handler: jsii.String("index.handler"),
		Code:    lambda.Code_FromAsset(jsii.String(filepath.Join(baseDir, "code")), nil),
	})

	fn.AddLayers(layer.NewKubectlV25Layer(stack, jsii.String("KubectlV25Layer")))

	return stack
}

This is the synthesized template:

{
 "Resources": {
  .... 
  "MyFunction3BAA72D1": {
   "Type": "AWS::Lambda::Function",
   "Properties": {
    "Code": {
     "S3Bucket": {
      "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
     },
     "S3Key": "6496abe9c6f32f60e58f0e56455ba87c91b27f052e4e9fdfa642ff85339a2521.zip"
    },
    "Role": {
     "Fn::GetAtt": [
      "MyFunctionServiceRole3C357FF2",
      "Arn"
     ]
    },
    "Handler": "index.handler",
    "Layers": [
     {
      "Ref": "KubectlV25Layer182E5352"
     }
    ],
    "Runtime": "python3.9"
   },
   "DependsOn": [
    "MyFunctionServiceRole3C357FF2"
   ],
   "Metadata": {
    "aws:cdk:path": "AppKubectlGoStack/MyFunction/Resource",
    "aws:asset:path": "asset.6496abe9c6f32f60e58f0e56455ba87c91b27f052e4e9fdfa642ff85339a2521",
    "aws:asset:is-bundled": false,
    "aws:asset:property": "Code"
   }
  },
  "KubectlV25Layer182E5352": {
   "Type": "AWS::Lambda::LayerVersion",
   "Properties": {
    "Content": {
     "S3Bucket": {
      "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
     },
     "S3Key": "ebc48a9a74e277f463f78c5a0b3bcf489a891efe25ece75b5b439b9e89bee4ac.zip"
    },
    "Description": "/opt/kubectl/kubectl 1.25; /opt/helm/helm 3.11",
    "LicenseInfo": "Apache-2.0"
   },
   "Metadata": {
    "aws:cdk:path": "AppKubectlGoStack/KubectlV25Layer/Resource",
    "aws:asset:path": "asset.ebc48a9a74e277f463f78c5a0b3bcf489a891efe25ece75b5b439b9e89bee4ac.zip",
    "aws:asset:is-bundled": false,
    "aws:asset:property": "Content"
   }
  }
  .....
 }
}

moelasmar · 2024-02-21T00:18:28Z

If you want to use the latest supported Python version (python 3.12), you can upgrade the aws-cdk-go library, and use the latest version (v2.128.0) and keep using old awscdk-kubectl-go library, and then you will be able to use Python 3.12 for your Lambda function.

See the following snippets:

\\ go.mod

module app_kubectl_go

go 1.18

require (
	github.com/aws/aws-cdk-go/awscdk/v2 v2.128.0
	github.com/aws/constructs-go/constructs/v10 v10.1.33
	github.com/aws/jsii-runtime-go v1.76.0
	github.com/cdklabs/awscdk-kubectl-go/kubectlv25/v2 v2.0.4
)

// app_kubectl_go.go
....
func NewAppKubectlGoStack(scope constructs.Construct, id string, props *AppKubectlGoStackProps) awscdk.Stack {
	var sprops awscdk.StackProps
	if props != nil {
		sprops = props.StackProps
	}
	stack := awscdk.NewStack(scope, &id, &sprops)
	baseDir, err := os.Getwd()

	if err != nil {
		fmt.Println("Failed to get current working directory", err)
		return nil
	}

	fn := lambda.NewFunction(stack, jsii.String("MyFunction"), &lambda.FunctionProps{
		Runtime: lambda.Runtime_PYTHON_3_12(),
		Handler: jsii.String("index.handler"),
		Code:    lambda.Code_FromAsset(jsii.String(filepath.Join(baseDir, "code")), nil),
	})

	fn.AddLayers(layer.NewKubectlV25Layer(stack, jsii.String("KubectlV25Layer")))

	return stack
}

synthesized template:

{
 "Resources": {
  "MyFunction3BAA72D1": {
   "Type": "AWS::Lambda::Function",
   "Properties": {
    "Code": {
     "S3Bucket": {
      "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
     },
     "S3Key": "6496abe9c6f32f60e58f0e56455ba87c91b27f052e4e9fdfa642ff85339a2521.zip"
    },
    "Handler": "index.handler",
    "Layers": [
     {
      "Ref": "KubectlV25Layer182E5352"
     }
    ],
    "Role": {
     "Fn::GetAtt": [
      "MyFunctionServiceRole3C357FF2",
      "Arn"
     ]
    },
    "Runtime": "python3.12"
   },
   "DependsOn": [
    "MyFunctionServiceRole3C357FF2"
   ],
   "Metadata": {
    "aws:cdk:path": "AppKubectlGoStack/MyFunction/Resource",
    "aws:asset:path": "asset.6496abe9c6f32f60e58f0e56455ba87c91b27f052e4e9fdfa642ff85339a2521",
    "aws:asset:is-bundled": false,
    "aws:asset:property": "Code"
   }
  },
  "KubectlV25Layer182E5352": {
   "Type": "AWS::Lambda::LayerVersion",
   "Properties": {
    "Content": {
     "S3Bucket": {
      "Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
     },
     "S3Key": "ebc48a9a74e277f463f78c5a0b3bcf489a891efe25ece75b5b439b9e89bee4ac.zip"
    },
    "Description": "/opt/kubectl/kubectl 1.25; /opt/helm/helm 3.11",
    "LicenseInfo": "Apache-2.0"
   },
   "Metadata": {
    "aws:cdk:path": "AppKubectlGoStack/KubectlV25Layer/Resource",
    "aws:asset:path": "asset.ebc48a9a74e277f463f78c5a0b3bcf489a891efe25ece75b5b439b9e89bee4ac.zip",
    "aws:asset:is-bundled": false,
    "aws:asset:property": "Content"
   }
  }
 }
}

moelasmar · 2024-02-21T00:22:29Z

@jaredhancock31, I hope my answers help address your concerns.

jaredhancock31 · 2024-02-21T15:35:54Z

re: your first reply - we don't make our own lambda layer , we just import the AWS-published ones and plug it into the cluster construct. Up til now there was never a need to implement our own kubectl layer in order for EKS stacks to function.

The guidance we got was to upgrade to the latest CDK, but unfortunately that's a large jump for us and it will take some time to fully test all the changes in latest. In general it was just a surprise to us that the code we import from AWS repos suddenly stopped working.

jaredhancock31 · 2024-02-21T16:52:41Z

@moelasmar we're going to proceed in trying to get caught up to latest CDK. Not sure how we can avoid situations like this in the future, but I suppose since there's a path forward we can close this.

moelasmar · 2024-02-21T19:21:47Z

you can still use the latest python runtime without upgrading your CDK version

runtime := lambda.NewRuntime(jsii.String("python3.12"), lambda.RuntimeFamily_PYTHON, &lambda.LambdaRuntimeProps{
		SupportsInlineCode:        jsii.Bool(true),
		SupportsCodeGuruProfiling: jsii.Bool(true),
	})

	fn := lambda.NewFunction(stack, jsii.String("MyFunction"), &lambda.FunctionProps{
		Runtime: runtime,
		Handler: jsii.String("index.handler"),
		Code:    lambda.Code_FromAsset(jsii.String(filepath.Join(baseDir, "code")), nil),
	})

moelasmar · 2024-02-21T19:25:38Z

Thanks @jaredhancock31 for raising this issue. I will mark this issue as closed. If you have any more concerns or questions, please open a new issue.

github-actions · 2024-02-21T19:25:55Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

jaredhancock31 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 15, 2024

github-actions bot added the @aws-cdk/aws-lambda Related to AWS Lambda label Feb 15, 2024

pahud added p1 effort/medium Medium work item – several days of effort and removed needs-triage This issue or PR still needs to be triaged. labels Feb 16, 2024

moelasmar self-assigned this Feb 21, 2024

moelasmar added cause/not-a-bug Not a bug (might still be a documentation issue, might still need work) and removed bug This issue is a bug. labels Feb 21, 2024

moelasmar added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Feb 21, 2024

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Feb 21, 2024

moelasmar closed this as completed Feb 21, 2024

github-actions bot mentioned this issue Mar 1, 2024

Monthly issue metrics report #29327

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

awscdk-kubectl-go: KubectlLayer modules out of date, broken by Python 3.7 runtime removal #29123

awscdk-kubectl-go: KubectlLayer modules out of date, broken by Python 3.7 runtime removal #29123

jaredhancock31 commented Feb 15, 2024 •

edited

Loading

pahud commented Feb 16, 2024 •

edited

Loading

moelasmar commented Feb 21, 2024

moelasmar commented Feb 21, 2024

moelasmar commented Feb 21, 2024

jaredhancock31 commented Feb 21, 2024

jaredhancock31 commented Feb 21, 2024

moelasmar commented Feb 21, 2024

moelasmar commented Feb 21, 2024

github-actions bot commented Feb 21, 2024

awscdk-kubectl-go: KubectlLayer modules out of date, broken by Python 3.7 runtime removal #29123

awscdk-kubectl-go: KubectlLayer modules out of date, broken by Python 3.7 runtime removal #29123

Comments

jaredhancock31 commented Feb 15, 2024 • edited Loading

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

CDK CLI Version

Framework Version

Node.js Version

OS

Language

Language Version

Other information

pahud commented Feb 16, 2024 • edited Loading

moelasmar commented Feb 21, 2024

moelasmar commented Feb 21, 2024

moelasmar commented Feb 21, 2024

jaredhancock31 commented Feb 21, 2024

jaredhancock31 commented Feb 21, 2024

moelasmar commented Feb 21, 2024

moelasmar commented Feb 21, 2024

github-actions bot commented Feb 21, 2024

⚠️COMMENT VISIBILITY WARNING⚠️

jaredhancock31 commented Feb 15, 2024 •

edited

Loading

pahud commented Feb 16, 2024 •

edited

Loading