Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple label selection ability in EtcdNodeSelectorLabels #5321

Merged
merged 1 commit into from
Sep 10, 2024

Conversation

tiansuo114
Copy link
Contributor

@tiansuo114 tiansuo114 commented Aug 7, 2024

What type of PR is this?
/kind feature

What this PR does / why we need it:
Supports the function of --etcd-node-selector-labels for multiple labels of nodes in the cluster

user@virtual-machine:karmada/_output/bin/linux/amd64# ./karmadactl init --etcd-node-selector-labels karmada.io/etcd=true,kubernetes.io/os=linux
I0830 09:58:51.083493    7186 deploy.go:111] No default release version found. build version: version.Info{GitVersion:"", GitCommit:"0c4ff310f2d3bba4c54a513c62bc4110e5353b32", GitTreeState:"clean", BuildDate:"2024-08-30T01:35:17Z", GoVersion:"go1.22.6", Compiler:"gc", Platform:"linux/amd64"}
I0830 09:58:51.083581    7186 enable_option.go:85] No default release version found. build version: version.Info{GitVersion:"", GitCommit:"0c4ff310f2d3bba4c54a513c62bc4110e5353b32", GitTreeState:"clean", BuildDate:"2024-08-30T01:35:17Z", GoVersion:"go1.22.6", Compiler:"gc", Platform:"linux/amd64"}
......
I0830 10:01:43.161625    7186 deploy.go:502] Create karmada webhook Deployment
I0830 10:01:43.167256    7186 idempotency.go:296] Service karmada-system/karmada-webhook has been created or updated.

------------------------------------------------------------------------------------------------------
 █████   ████   █████████   ███████████   ██████   ██████   █████████   ██████████     █████████
░░███   ███░   ███░░░░░███ ░░███░░░░░███ ░░██████ ██████   ███░░░░░███ ░░███░░░░███   ███░░░░░███
 ░███  ███    ░███    ░███  ░███    ░███  ░███░█████░███  ░███    ░███  ░███   ░░███ ░███    ░███
 ░███████     ░███████████  ░██████████   ░███░░███ ░███  ░███████████  ░███    ░███ ░███████████
 ░███░░███    ░███░░░░░███  ░███░░░░░███  ░███ ░░░  ░███  ░███░░░░░███  ░███    ░███ ░███░░░░░███
 ░███ ░░███   ░███    ░███  ░███    ░███  ░███      ░███  ░███    ░███  ░███    ███  ░███    ░███
 █████ ░░████ █████   █████ █████   █████ █████     █████ █████   █████ ██████████   █████   █████
░░░░░   ░░░░ ░░░░░   ░░░░░ ░░░░░   ░░░░░ ░░░░░     ░░░░░ ░░░░░   ░░░░░ ░░░░░░░░░░   ░░░░░   ░░░░░
------------------------------------------------------------------------------------------------------
Karmada is installed successfully.

Register Kubernetes cluster to Karmada control plane.

Register cluster with 'Push' mode

Step 1: Use "karmadactl join" command to register the cluster to Karmada control plane. --cluster-kubeconfig is kubeconfig of the member cluster.
(In karmada)~# MEMBER_CLUSTER_NAME=$(cat ~/.kube/config  | grep current-context | sed 's/: /\n/g'| sed '1d')
(In karmada)~# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config  join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config

Step 2: Show members of karmada
(In karmada)~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters


Register cluster with 'Pull' mode

Step 1: Use "karmadactl register" command to register the cluster to Karmada control plane. "--cluster-name" is set to cluster of current-context by default.
(In member cluster)~# karmadactl register 172.18.0.2:32443 --token ofc912.rk5yj36jfc9es9by --discovery-token-ca-cert-hash sha256:82c3150ef0593fe5f581078cb6ccfc597a0043a45ff2af028b302d1cd53ad9f9

Step 2: Show members of karmada
(In karmada)~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters

user@virtual-machine:karmada/_output/bin/linux/amd64#

Which issue(s) this PR fixes:
Fixes #5320
This PR is related to the documentation PR: #5277
Special notes for your reviewer:
@liangyuanpeng
Does this PR introduce a user-facing change?:

karmadactl: support multiple label selection ability with flag EtcdNodeSelectorLabels.

@liangyuanpeng
Copy link
Contributor

liangyuanpeng commented Aug 7, 2024

Hi, this is a bot comment, just put the label of ok-to-test here, you can comment /retest to rerun github workflow if github workflow is failing and it's not releated with this PR.

/ok-to-test

@karmada-bot karmada-bot added the kind/feature Categorizes issue or PR as related to a new feature. label Aug 7, 2024
@karmada-bot karmada-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 7, 2024
@liangyuanpeng
Copy link
Contributor

/ok-to-test

@codecov-commenter
Copy link

codecov-commenter commented Aug 7, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 22.22222% with 14 lines in your changes missing coverage. Please review.

Project coverage is 31.68%. Comparing base (e13518d) to head (2c57eba).
Report is 81 commits behind head on master.

Files with missing lines Patch % Lines
pkg/karmadactl/cmdinit/kubernetes/deploy.go 0.00% 12 Missing ⚠️
pkg/karmadactl/cmdinit/kubernetes/statefulset.go 60.00% 1 Missing and 1 partial ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5321      +/-   ##
==========================================
+ Coverage   30.97%   31.68%   +0.71%     
==========================================
  Files         637      643       +6     
  Lines       44266    44456     +190     
==========================================
+ Hits        13712    14088     +376     
+ Misses      29563    29337     -226     
- Partials      991     1031      +40     
Flag Coverage Δ
unittests 31.68% <22.22%> (+0.71%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tiansuo114
Copy link
Contributor Author

/retest

1 similar comment
@tiansuo114
Copy link
Contributor Author

/retest

Comment on lines 553 to 563
func mapsEqual(a, b map[string]string) bool {
if len(a) != len(b) {
return false
}
for k, v := range a {
if b[k] != v {
return false
}
}
return true
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just use reflect.DeepEqual(m1,m2), so this is not needed

Copy link
Contributor

@liangyuanpeng liangyuanpeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a description of what this PR does and why it is needed so that other reviewers can understand the PR.

/cc @zhzhuang-zju for help to review,thanks.

Copy link
Contributor

@liangyuanpeng liangyuanpeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

(need to squash commits before merge)

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Aug 9, 2024
@karmada-bot karmada-bot removed the lgtm Indicates that a PR is ready to be merged. label Aug 11, 2024
@tiansuo114
Copy link
Contributor Author

/lgtm

(need to squash commits before merge)

Probably because I made a mistake with the rabase operation, I might need to review the code again😭😭

@liangyuanpeng
Copy link
Contributor

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Aug 12, 2024
@tiansuo114
Copy link
Contributor Author

Add a description of what this PR does and why it is needed so that other reviewers can understand the PR.

/cc @zhzhuang-zju for help to review,thanks.

Description for review:

Design Purpose

In certain scenarios, users may need to deploy Etcd on nodes with specific attributes, such as operating system type, hardware architecture, or custom labels. By allowing multiple labels to be specified, users can more accurately control the deployment of Etcd, thereby improving system robustness and performance.

Implementation Details

Command Flag: Users can specify multiple labels using the --EtcdNodeSelectorLabels flag. The format for labels is key1=value1,key2=value2, with each label separated by a comma.

  • Label Parsing: A new function parseEtcdNodeSelectorLabelsMap was introduced to parse the label string in EtcdNodeSelectorLabels into a key-value pair map (map[string]string) for subsequent use in the node selector.
  • Node Validation: In the Complete() function, the program iterates through each label to check if there are nodes in the cluster that match the criteria. If no matching nodes are found, an error message is returned.
  • StatefulSet Configuration: When creating the Etcd StatefulSet, the makeETCDStatefulSet function uses the parsed label map to set the node selector.

Usage

Users can specify the EtcdNodeSelectorLabels as follows when using the karmadactl init command:
karmadactl init --EtcdNodeSelectorLabels="kubernetes.io/os=linux,hello=world"
This ensures that Etcd is only deployed on nodes that have both the kubernetes.io/os=linux and hello=world labels.

Testing and Validation

To ensure the correctness of this feature, corresponding unit tests have been added, including:

  • Valid Label Parsing Tests: Tests whether multiple correctly formatted labels can be properly parsed into a map.
  • Invalid Label Tests: Tests whether labels without an equal sign or with incorrect formatting trigger an appropriate error.
  • Label Application Tests: Tests whether the parsed labels are correctly applied to the Etcd StatefulSet configuration.

@tiansuo114
Copy link
Contributor Author

This PR is related to the documentation PR: #5277

@liangyuanpeng
Copy link
Contributor

for reviewers, this PR is coming from #5277 (comment)

@liangyuanpeng
Copy link
Contributor

liangyuanpeng commented Aug 20, 2024

@hulizhe you may interested in

@XiShanYongYe-Chang
Copy link
Member

/assign @zhzhuang-zju

Comment on lines 281 to 285
labels := strings.Split(i.EtcdNodeSelectorLabels, ",")
for _, label := range labels {
if !i.isNodeExist(label) {
return fmt.Errorf("no node found by label %s", label)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
labels := strings.Split(i.EtcdNodeSelectorLabels, ",")
for _, label := range labels {
if !i.isNodeExist(label) {
return fmt.Errorf("no node found by label %s", label)
}
if !i.isNodeExist(i.EtcdNodeSelectorLabels) {
return fmt.Errorf("no node found by label %s", i.EtcdNodeSelectorLabels)

Here, we should verify if there are any nodes that match all the labels specified in a single string like "key1=value1,key2=value2", so we should not split the string into separate conditions.

@zhzhuang-zju
Copy link
Contributor

@tiansuo114

flags.StringVarP(&opts.EtcdNodeSelectorLabels, "etcd-node-selector-labels", "", "", "etcd pod select the labels of the node. valid in hostPath mode ( e.g. --etcd-node-selector-labels karmada.io/etcd=true)")

We can refine the description of the flag etcd-node-selector-labels to clarify its purpose more clearly, and add a release note

@karmada-bot karmada-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 3, 2024
@liangyuanpeng
Copy link
Contributor

/lgtm
/cc @zhzhuang-zju if you want to lgtm again.
/assign @XiShanYongYe-Chang @chaunceyjiang
for approval

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 3, 2024
@XiShanYongYe-Chang
Copy link
Member

Hi @tiansuo114, can you help add a release note?

@tiansuo114
Copy link
Contributor Author

tiansuo114 commented Sep 3, 2024

Hi @tiansuo114, can you help add a release note?您好,您能帮忙添加发行说明吗?

What should I do? Is it related to modifications to the karmadactl documentation? If so, I think it can be included in the final supplementary documentation phase of my OSPP task

@liangyuanpeng
Copy link
Contributor

liangyuanpeng commented Sep 3, 2024

Add the description into the Does this PR introduce a user-facing change?: of your PR body.

Maybe: karmadactl: support multiple label selection ability with flag EtcdNodeSelectorLabels.

@tiansuo114
Copy link
Contributor Author

Add the description into the Does this PR introduce a user-facing change?: of your PR body.将描述添加到 PR 正文的 Does this PR introduce a user-facing change?: 中。

Maybe: karmadactl: support multiple label selection ability with flag EtcdNodeSelectorLabels. 也许: karmadactl: support multiple label selection ability with flag EtcdNodeSelectorLabels.

Okay, I put it in the pr body

if err != nil {
return fmt.Errorf("failed to convert etcdNodeSelector labels to map: %v", err)
}
i.EtcdNodeSelectorLabelsMap = labelMap
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question, and the variable setting logic is involved here, is it more reasonable to put it in the Complete function?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is make sense,I'm glad we're getting closer and closer to merge this PR.

Comment on lines 302 to 306
if i.EtcdStorageMode == "hostPath" && i.EtcdNodeSelectorLabelsMap != nil {
podSpec.NodeSelector = i.EtcdNodeSelectorLabelsMap
}
if i.EtcdStorageMode == "hostPath" && i.EtcdNodeSelectorLabels == "" {
podSpec.NodeSelector = map[string]string{"karmada.io/etcd": ""}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about update like this:

if i.EtcdStorageMode == "hostPath" {
		if i.EtcdNodeSelectorLabelsMap != nil {
			podSpec.NodeSelector = i.EtcdNodeSelectorLabelsMap
		} else {
			podSpec.NodeSelector = map[string]string{"karmada.io/etcd": ""}
		}
	}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually better, I fixed the above problem, now how does the code look

Copy link
Member

@XiShanYongYe-Chang XiShanYongYe-Chang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. It's a meaningful modification.

Signed-off-by: tiansuo114 <1729765480@qq.com>

fix ci

Signed-off-by: tiansuo114 <1729765480@qq.com>

add unit test

Signed-off-by: tiansuo114 <1729765480@qq.com>

change unit test

Signed-off-by: tiansuo114 <1729765480@qq.com>

rm mapsEqual

Signed-off-by: tiansuo114 <1729765480@qq.com>

fix

Signed-off-by: tiansuo114 <1729765480@qq.com>

1111111111111111111111

Signed-off-by: tiansuo114 <1729765480@qq.com>

Update pkg/karmadactl/cmdinit/cmdinit.go

Co-authored-by: zhzhuang-zju <m17799853869@163.com>

use metav1.ParseToLabelSelector

Signed-off-by: tiansuo114 <1729765480@qq.com>

delete useless test

Signed-off-by: tiansuo114 <1729765480@qq.com>

11

Signed-off-by: tiansuo114 <1729765480@qq.com>

fix

Signed-off-by: tiansuo114 <1729765480@qq.com>
@liangyuanpeng
Copy link
Contributor

liangyuanpeng commented Sep 10, 2024

kindly ping @XiShanYongYe-Chang

@XiShanYongYe-Chang
Copy link
Member

Thanks~
/approve

@karmada-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: XiShanYongYe-Chang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@karmada-bot karmada-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 10, 2024
@zhzhuang-zju
Copy link
Contributor

/lgtm

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 10, 2024
@karmada-bot karmada-bot merged commit 4668cf0 into karmada-io:master Sep 10, 2024
12 checks passed
@tiansuo114 tiansuo114 deleted the node branch November 23, 2024 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. ok-to-test size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support multiple label selection ability in karmadactl init
7 participants