performance improvement for list request with labelSelector

[likakuli]

What would you like to be added:

Use index to improve the performance of list request with labelSelector

Why is this needed:

LabelSelector is used in many places in karmada-controller to list object. controller-runtime not setup index for label and not support to setup index for label too. So the list request cost O(n). When there are many object, it will cost too much time.

controller-runtime code: https://github.com/kubernetes-sigs/controller-runtime/blob/f46919744bee01060c9084a285e049afffd38c9d/pkg/cache/internal/cache_reader.go#L115-L138

[likakuli]

I don't know whether this function should be implemented in controller-runtime or just in karmada-controller.
There are two points related to this.

1. controller-runtime not support index when to use labelSelector in list request.
2. controller-runtime and client-go not support complicated fieldSelector just as the comment.

In our case, we implemented this by two steps.

1. We add custom manager and cacher used to convert labelSelector to fieldSelector.
2. We use custom client-go and controller-runtime package to support complicated selector. refer to:
   Thread safe store support complicated index kubernetes/kubernetes#109334
   ✨ Support multiple field selectors kubernetes-sigs/controller-runtime#1838

[RainbowMango]

cc @yy158775 Please take a look this issue.

[yy158775]

From my point of view,I think we can convet labelSelector to fieldSelector with a custom IndexFunc.The IndexFunc can compute the set of indexed values for an object's labels.

Eg: If we want to select resourcebindings with label key "clusterpropagationpolicy.karmada.io/name" = specific value,we can construct a IndexFunc which return "key=value" pair when the "clusterpropagationpolicy.karmada.io/name" exists in object labels.

Such as:
indices["labels"] = {"clusterpropagationpolicy.karmada.io/name=$var":[object_key1,object_key2,object_key3,......],......}
indexers["labels"] = indexfunc

If we need to select objects by other labels,we can edit the indexfunc to support.

[likakuli]

If no one is working on it, i can open a pr to support this issue.

[RainbowMango]

Sure thing! (anyone is welcome to pick a task without an assignment)

Have you done it on your side?

I know @yy158775 started looking at it and working on the demo.

This issue is much more involved. And @yy158775 can cooperate with you.

[likakuli]

Our internal version has implemented this function.

[yy158775]

I think this is worth optimizing.So I look foward to seeing your version.

[RainbowMango]

/assign @likakuli
So, please help to make a PR for this.

[yy158775]

Our internal version has implemented this function.

Are you convenient to open a PR? I am ready to work on this issue with you.

[likakuli]

sorry, i will open a pr this weekend. or you can open a pr if you have already completed it. thx

[weilaaa]

In my opinion, we need to fundamentally enhance the capabilities of client-go. See proposal issues: kubernetes/kubernetes#109329

app-->controller-runtime-->client-go this is fine.

[RainbowMango]

@weilaaa Are you suggesting waiting for the proposed issue in K/K?

[weilaaa]

Yes, it may be the most appropriate way, but I can't guarantee that this proposal will be passed. At least we can wait for the conclusion of the community(K/K) on this proposal.