Failed to Create Karmada Instance of the operator due to CRD Download Error (404)

[mohamedawnallah]

What happened:
While following the installation guide from operator/README.md, I attempted to install the Karmada Operator with the following command:

helm install karmada-operator -n karmada-system --create-namespace --dependency-update ./charts/karmada-operator --debug

The Karmada Operator was successfully deployed, as shown by the status output:

$ helm status karmada-operator -n karmada-system
NAME: karmada-operator
LAST DEPLOYED: Sat Sep 21 07:49:41 2024
NAMESPACE: karmada-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

However, when creating a Karmada instance with:

kubectl create namespace test
kubectl apply -f operator/config/samples/karmada.yaml

I encountered the following error:

failed to download CRD tar, err: failed download file. url: https://github.com/karmada-io/karmada/releases/download/v1.12.0-alpha.0/crds.tar.gz code: 404

part of:

apiVersion: v1
items:
  - apiVersion: operator.karmada.io/v1alpha1
    kind: Karmada
    metadata:
      annotations:
        kubectl.kubernetes.io/last-applied-configuration: |
          {"apiVersion":"operator.karmada.io/v1alpha1","kind":"Karmada","metadata":{"annotations":{},"name":"karmada-demo","namespace":"test"},"spec":{"components":{"etcd":{"local":{"imageRepository":"registry.k8s.io/etcd","imageTag":"3.5.13-0","replicas":1,"volumeData":{"volumeClaim":{"metadata":{"name":"etcd-data"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"3Gi"}}}}}}},"karmadaAPIServer":{"imageRepository":"registry.k8s.io/kube-apiserver","imageTag":"v1.30.4","replicas":1,"serviceSubnet":"10.96.0.0/12","serviceType":"NodePort"},"karmadaAggregatedAPIServer":{"imageRepository":"docker.io/karmada/karmada-aggregated-apiserver","imageTag":"v1.8.0","replicas":1},"karmadaControllerManager":{"imageRepository":"docker.io/karmada/karmada-controller-manager","imageTag":"v1.8.0","replicas":1},"karmadaMetricsAdapter":{"imageRepository":"docker.io/karmada/karmada-metrics-adapter","imageTag":"v1.8.0","replicas":2},"karmadaScheduler":{"imageRepository":"docker.io/karmada/karmada-scheduler","imageTag":"v1.8.0","replicas":1},"karmadaWebhook":{"imageRepository":"docker.io/karmada/karmada-webhook","imageTag":"v1.8.0","replicas":1},"kubeControllerManager":{"imageRepository":"registry.k8s.io/kube-controller-manager","imageTag":"v1.30.4","replicas":1}},"hostCluster":{"networking":{"dnsDomain":"cluster.local"}}}}
      creationTimestamp: "2024-09-21T04:46:59Z"
      finalizers:
        - operator.karmada.io/finalizer
      generation: 2
      labels:
        operator.karmada.io/disable-cascading-deletion: "false"
      name: karmada-demo
      namespace: test
      resourceVersion: "3349"
      uid: f736885e-1cff-4312-97f4-89f7ac9d0ed5
    spec:
      components:
        etcd:
          local:
            imagePullPolicy: IfNotPresent
            imageRepository: registry.k8s.io/etcd
            imageTag: 3.5.13-0
            replicas: 1
            resources: {}
            volumeData:
              volumeClaim:
                metadata:
                  name: etcd-data
                spec:
                  accessModes:
                    - ReadWriteOnce
                  resources:
                    requests:
                      storage: 3Gi
        karmadaAPIServer:
          imagePullPolicy: IfNotPresent
          imageRepository: registry.k8s.io/kube-apiserver
          imageTag: v1.30.4
          replicas: 1
          resources: {}
          serviceSubnet: 10.96.0.0/12
          serviceType: NodePort
        karmadaAggregatedAPIServer:
          imagePullPolicy: IfNotPresent
          imageRepository: docker.io/karmada/karmada-aggregated-apiserver
          imageTag: v1.8.0
          replicas: 1
          resources: {}
        karmadaControllerManager:
          imagePullPolicy: IfNotPresent
          imageRepository: docker.io/karmada/karmada-controller-manager
          imageTag: v1.8.0
          replicas: 1
          resources: {}
        karmadaMetricsAdapter:
          imagePullPolicy: IfNotPresent
          imageRepository: docker.io/karmada/karmada-metrics-adapter
          imageTag: v1.8.0
          replicas: 2
          resources: {}
        karmadaScheduler:
          imagePullPolicy: IfNotPresent
          imageRepository: docker.io/karmada/karmada-scheduler
          imageTag: v1.8.0
          replicas: 1
          resources: {}
        karmadaWebhook:
          imagePullPolicy: IfNotPresent
          imageRepository: docker.io/karmada/karmada-webhook
          imageTag: v1.8.0
          replicas: 1
          resources: {}
        kubeControllerManager:
          imagePullPolicy: IfNotPresent
          imageRepository: registry.k8s.io/kube-controller-manager
          imageTag: v1.30.4
          replicas: 1
          resources: {}
      hostCluster:
        networking:
          dnsDomain: cluster.local
    status:
      conditions:
        - lastTransitionTime: "2024-09-21T05:00:59Z"
          message: 'failed to download CRD tar, err: failed download file. url: https://github.com/karmada-io/karmada/releases/download/v1.12.0-alpha.0/crds.tar.gz code: 404'
          reason: Failed
          status: "False"
          type: Ready
kind: List
metadata:
  resourceVersion: ""

What you expected to happen:
The Karmada instance should have been created successfully using the provided sample file.

How to reproduce it (as minimally and precisely as possible):
Please follow the installation guide steps and attempt to create a Karmada instance using the sample file.

Anything else we need to know?:
Please let me know if I'm missing something it may be simple and also the kind/bug may not be suitable for this but I found it has a structured PR template for my case here. Thanks! 🙏

Environment:

• Karmada version: master branch

[chaosi-zju]

hi, can you try modify following line:

karmada/charts/karmada-operator/values.yaml

Lines 46 to 49 in 6c484ca

	image:
	registry: docker.io
	repository: karmada/karmada-operator
	tag: latest

update tag: latest to tag: v1.10.0, and then do the following steps?

[chaosi-zju]

It can't be said to be a bug, perhaps not very well documented.

Actually, the version of crd downloaded by karmada-operator depends on the gitVersion ldflags of karmada-operator own binary artifact.

You can simply understand it as the released version karmada-operator:v1.10.0 will download v1.10.0 version crd.

However, karmada-operator:latest is built by latest code from master branch with a unreleased version (like v1.12.0-alpha.0-xx-xxxxxxxxxx), and it will try to download a unreleased version crd (like: v1.12.0-alpha.0), but the corresponding crd link doesn't exist. So, there is indeed a problem if you use karmada-operator:latest to install Karmada.

[mohamedawnallah]

Thanks, @chaosi-zju, for your response! It definitely helps! 🙏. I'm willing to submit a PR to update the docs. Could we generalize this by guiding users to automatically use the latest release version? As time of writing this, it would be v1.11.1.

[chaosi-zju]

As time of writing this, it would be v1.11.1.

Yes, but now the v1.11.0+ version of karmada-operator has a bug, and is on the way to fix, the related PR is #5519. That's why I didn't ask you to install v1.11.0 when I gave the example.

[chaosi-zju]

I'm willing to submit a PR to update the docs. Could we generalize this by guiding users to automatically use the latest release version?

Thank you so much for your interest in contributing to improve it, welcome, your contributions will make a big difference!

As for how to improve it, it seems there are two ways:

• update the docs to guide users to use the latest release version.
• update the operator's logic, when it handles the unreleased version, let it automatically download the latest release version crd

@RainbowMango which do you think is better?

[zhzhuang-zju]

Yes, but now the v1.11.0+ version of karmada-operator has a bug, and is on the way to fix, the related PR is #5519.

this issue has been fixed in #5546

@chaosi-zju @mohamedawnallah In the release-1.11 version, we provided the ability to customize the CRD URL for the Karmada Operator, see #5185. Therefore, we can also resolve this issue by adding the download path for the CRDs in the karmada.yaml file.

  crdTarball:
    httpSource:
      url: https://github.com/karmada-io/karmada/releases/download/v1.11.0/crds.tar.gz

[chaosi-zju]

this issue has been fixed in #5546

Oh, okey.

[RainbowMango]

update the docs to guide users to use the latest release version.
update the operator's logic, when it handles the unreleased version, let it automatically download the latest release version crd
@RainbowMango which do you think is better?

Update the docs.

[chaosi-zju]

so I'm willing to submit a PR to update the docs. Could we generalize this by guiding users to automatically use the latest release version? As time of writing this, it would be v1.11.1.

hi @mohamedawnallah, so let's do it by update the docs to guide users using the latest release version, looking forward to your PR~

[mohamedawnallah]

Thanks @chaosi-zju! I've just submitted a PR for this. I would love to receive any feedback! 🙏