k8s 1.23 and 1.24 not working with cert-manager v1.0.2 and aws-load-balancer-controller 2.2.1

[ensean]

Hello, in the guide 步骤3-部署配置aws-load-balancer-controller&2048游戏 cert-manager is pinned to v1.0.2 and aws-loadbalancer-controller is pinned to v2.2.1 with the k8s version in this guide series is pinned to 1.21.

As 1.21 will be EOL soon, we follow the guide but with k8s version set to 1.23/1.24, and we found that under this two versions, cert-manager v1.0.2 and aws-loadbalancer-controller v2.2.1 can not work properly

[ec2-user@ip-172-31-29-13 ~]$ kubectl apply -f v2_2_1_full.yaml
customresourcedefinition.apiextensions.k8s.io/ingressclassparams.elbv2.k8s.aws created
customresourcedefinition.apiextensions.k8s.io/targetgroupbindings.elbv2.k8s.aws created
role.rbac.authorization.k8s.io/aws-load-balancer-controller-leader-election-role created
clusterrole.rbac.authorization.k8s.io/aws-load-balancer-controller-role created
rolebinding.rbac.authorization.k8s.io/aws-load-balancer-controller-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/aws-load-balancer-controller-rolebinding created
service/aws-load-balancer-webhook-service created
deployment.apps/aws-load-balancer-controller created
mutatingwebhookconfiguration.admissionregistration.k8s.io/aws-load-balancer-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/aws-load-balancer-webhook created
Error from server (InternalError): error when creating "v2_2_1_full.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": x509: certificate signed by unknown authority
Error from server (InternalError): error when creating "v2_2_1_full.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": x509: certificate signed by unknown authority

[ec2-user@ip-172-31-29-13 ~]$ kubectl get pods -A
NAMESPACE      NAME                                            READY   STATUS             RESTARTS      AGE
cert-manager   cert-manager-846ffbd788-bxghv                   1/1     Running            0             88s
cert-manager   cert-manager-cainjector-6798db999f-6bgp2        0/1     CrashLoopBackOff   3 (34s ago)   88s
cert-manager   cert-manager-webhook-6f96656b76-dxr9b           1/1     Running            0             88s
kube-system    aws-load-balancer-controller-5b4fbd597d-pf7c2   0/1     CrashLoopBackOff   3 (19s ago)   72s
kube-system    aws-node-czs8t                                  1/1     Running            0             31h
kube-system    aws-node-r5qsg                                  1/1     Running            0             31h
kube-system    coredns-5fc8d4cdcf-gfqkg                        1/1     Running            0             32h
kube-system    coredns-5fc8d4cdcf-vdmth                        1/1     Running            0             32h
kube-system    kube-proxy-jh77q                                1/1     Running            0             31h
kube-system    kube-proxy-qm82h                                1/1     Running            0             31h
[ec2-user@ip-172-31-29-13 ~]$ kubectl logs aws-load-balancer-controller-5b4fbd597d-pf7c2 -n kube-system
{"level":"info","ts":1675087496.721926,"msg":"version","GitVersion":"v2.2.1","GitCommit":"27803e3f8e3b637873f9bb59c56b78de01f65b79","BuildDate":"2021-06-25T17:18:28+0000"}
{"level":"info","ts":1675087496.8235195,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"error","ts":1675087496.831011,"logger":"setup","msg":"unable to create controller","controller":"Ingress","error":"the server could not find the requested resource"}
[ec2-user@ip-172-31-29-13 ~]$ kubectl logs cert-manager-cainjector-6798db999f-6bgp2 -n cert-manager
I0130 14:05:33.586910       1 start.go:89] "starting" version="v1.0.2" revision="219b7934ac499c7818526597cf635a922bddd22e"
I0130 14:05:35.437638       1 request.go:645] Throttling request took 1.040869167s, request: GET:https://10.100.0.1:443/apis/storage.k8s.io/v1beta1?timeout=32s
I0130 14:05:36.437651       1 request.go:645] Throttling request took 2.040297218s, request: GET:https://10.100.0.1:443/apis/elbv2.k8s.aws/v1beta1?timeout=32s
E0130 14:05:36.492648       1 start.go:131] cert-manager/ca-injector "msg"="error registering controllers" "error"="no matches for kind \"MutatingWebhookConfiguration\" in version \"admissionregistration.k8s.io/v1beta1\""

It works if we upgrade cert-manager to v1.11.0 and aws-load-balancer-controller to v2.4.6 when running k8s 1.23/1.24

As EKS is going the drop support for k8s 1.21 soon, is it possible to provide some kind of guide for the versions of k8s, cert-manager and aws-loadbalancer-controller?

Thanks a lot.