Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

keep ResourceBinding when removing PropagationPolicy #601

Merged
merged 1 commit into from
Aug 25, 2021

Conversation

qianjun1993
Copy link
Contributor

What type of PR is this?
/kind feature

What this PR does / why we need it:

stop delete resourcebinding when the related policy is deleted

Which issue(s) this PR fixes:
Fixes #505

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


@karmada-bot karmada-bot added kind/feature Categorizes issue or PR as related to a new feature. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 13, 2021
@RainbowMango
Copy link
Member

I haven't dig into why the E2E failing. Just echo the logs here and try again.

Export Service from source-clusters, import Service to destination-clusters
  /root/actions-runner/_work/karmada/karmada/test/e2e/mcs_test.go:346
STEP: Create ClusterPropagationPolicy(serviceexports-d92-policy) to Propagation ServiceExport CRD
STEP: Create ClusterPropagationPolicy(serviceimports-hwh-policy) to Propagation ServiceImport CRD
STEP: Wait ServiceExport CRD present on member clusters
I0813 15:16:46.647139  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:16:51.663366  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:16:56.664463  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:01.663626  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:06.663848  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:11.663597  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:16.663025  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:21.693098  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:26.663388  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:31.663187  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:36.663010  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:41.662953  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:46.670273  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:51.662679  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:17:56.662566  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:18:01.667378  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:18:06.662612  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
I0813 15:18:11.665766  598021 mcs_test.go:224] Waiting for CRD(ServiceExport) present on member cluster(member1)
STEP: Wait ServiceImport CRD present on member clusters
STEP: Create Deployment(karmadatest-5dl/hello-vp2) in member1 cluster
STEP: Create Service(karmadatest-5dl/hello-vp2) in member1 cluster
STEP: Wait Service(karmadatest-5dl/hello-vp2)'s EndpointSlice exist in member1 cluster
STEP: Create ServiceExport(karmadatest-5dl/hello-vp2)
STEP: Create PropagationPolicy(karmadatest-5dl/export-hello-vp2-policy) to propagate ServiceExport
STEP: Wait EndpointSlices collected to namespace(karmadatest-5dl) in controller-plane
STEP: Create ServiceImport(karmadatest-5dl/hello-vp2)
STEP: Create PropagationPolicy(karmadatest-5dl/import-hello-vp2-policy) to propagate ServiveImport
STEP: Wait derived-service(karmadatest-5dl/derived-hello-vp2) exist in member2 cluster
STEP: Wait EndpointSlices have been imported to member2 cluster
STEP: TCP connects across clusters using the ClusterIP
STEP: UDP connects across clusters using the ClusterIP
STEP: Cleanup
STEP: Delete Deployment(karmadatest-5dl/hello-vp2) in member1 cluster
STEP: Delete Service(karmadatest-5dl/hello-vp2) in member1 cluster
STEP: Delete ClusterPropagationPolicy serviceexports-d92-policy
STEP: Delete ClusterPropagationPolicy serviceimports-hwh-policy
STEP: Wait ServiceExport CRD disappear on member clusters

• Failure in Spec Teardown (AfterEach) [430.464 seconds]
[MCS] Multi-Cluster Service testing
/root/actions-runner/_work/karmada/karmada/test/e2e/mcs_test.go:178
  Connectivity testing [AfterEach]
  /root/actions-runner/_work/karmada/karmada/test/e2e/mcs_test.go:298
    Export Service from source-clusters, import Service to destination-clusters
    /root/actions-runner/_work/karmada/karmada/test/e2e/mcs_test.go:346

    Unexpected error:
        <*errors.errorString | 0xc0002227d0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

    /root/actions-runner/_work/karmada/karmada/test/e2e/mcs_test.go:278
------------------------------
SSSSSSSSSSSSSSSS

Summarizing 1 Failure:

[Fail] [MCS] Multi-Cluster Service testing [AfterEach] Connectivity testing Export Service from source-clusters, import Service to destination-clusters 
/root/actions-runner/_work/karmada/karmada/test/e2e/mcs_test.go:278

Ran 6 of 22 Specs in 466.684 seconds
FAIL! -- 5 Passed | 1 Failed | 0 Pending | 16 Skipped
--- FAIL: TestE2E (466.80s)
FAIL

@qianjun1993
Copy link
Contributor Author

Do you know when the crd of ServiceImport CRD is created. Is it created in advance? I only found the creation of ClusterPropagationPolicy in this test.

@XiShanYongYe-Chang
Copy link
Member

Do you know when the crd of ServiceImport CRD is created. Is it created in advance? I only found the creation of ClusterPropagationPolicy in this test.

The ServiceImport and ServiceExport CRD were created in karmada installation:

kubectl apply -f "${REPO_ROOT}/artifacts/deploy/multicluster.x-k8s.io_serviceexports.yaml"
kubectl apply -f "${REPO_ROOT}/artifacts/deploy/multicluster.x-k8s.io_serviceimports.yaml"

@XiShanYongYe-Chang
Copy link
Member

You may need to modify the e2e test:

ginkgo.By("Wait ServiceExport CRD disappear on member clusters", func() {
err := wait.PollImmediate(pollInterval, pollTimeout, func() (done bool, err error) {
clusters, err := fetchClusters(karmadaClient)
if err != nil {
return false, err
}
for _, cluster := range clusters {
if helper.IsAPIEnabled(cluster.Status.APIEnablements, mcsv1alpha1.GroupVersion.String(), util.ServiceExportKind) {
return false, nil
}
}
return true, nil
})
gomega.Expect(err).ShouldNot(gomega.HaveOccurred())
})
ginkgo.By("Wait ServiceImport CRD disappear on member clusters", func() {
err := wait.PollImmediate(pollInterval, pollTimeout, func() (done bool, err error) {
clusters, err := fetchClusters(karmadaClient)
if err != nil {
return false, err
}
for _, cluster := range clusters {
if helper.IsAPIEnabled(cluster.Status.APIEnablements, mcsv1alpha1.GroupVersion.String(), util.ServiceImportKind) {
return false, nil
}
}
return true, nil
})
gomega.Expect(err).ShouldNot(gomega.HaveOccurred())
})
})

the original logic will check whether ServiceExport and ServiceImport CRD has been deleted in member clusters.

@qianjun1993
Copy link
Contributor Author

So just remove this check?

@XiShanYongYe-Chang
Copy link
Member

So just remove this check?

I think so.

@karmada-bot karmada-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 23, 2021
@qianjun1993 qianjun1993 force-pushed the delete-policy branch 2 times, most recently from eb197b2 to ef67c09 Compare August 23, 2021 06:49
@RainbowMango
Copy link
Member

/assign
I'll review it ASAP, thanks.

@@ -258,41 +258,6 @@ var _ = ginkgo.Describe("[MCS] Multi-Cluster Service testing", func() {
err := karmadaClient.PolicyV1alpha1().ClusterPropagationPolicies().Delete(context.TODO(), serviceImportPolicy.Name, metav1.DeleteOptions{})
gomega.Expect(err).ShouldNot(gomega.HaveOccurred())
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some comment to record why not wait ServiceExport and ServiceImport disappear on member clusters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I will add it.

@@ -866,20 +866,14 @@ func (d *ResourceDetector) HandlePropagationPolicyDeletion(policyNS string, poli
return err
}

for itemIndex, binding := range rbs.Items {
for _, binding := range rbs.Items {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method's comments should be updated as well, e.g.

// HandlePropagationPolicyDeletion handles PropagationPolicy delete event.
// After a policy is removed, the label marked on relevant resource template will be removed(which gives
// the resource template a change to match another policy).
//
// Note: The relevant ResourceBinding will continue to exist until the resource template is gone.
func (d *ResourceDetector) HandlePropagationPolicyDeletion(policyNS string, policyName string) error {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I will change it.

@@ -894,48 +888,36 @@ func (d *ResourceDetector) HandleClusterPropagationPolicyDeletion(policyName str
policyv1alpha1.ClusterPropagationPolicyLabel: policyName,
}

// load and remove the ClusterResourceBindings which labeled with current policy
// load the ClusterResourceBindings which labeled with current policy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method's comments should be updated as well, e.g.:

// HandleClusterPropagationPolicyDeletion handles ClusterPropagationPolicy delete event.
// After a policy is removed, the label marked on relevant resource template will be removed(which gives
// the resource template a change to match another policy).
//
// Note: The relevant ClusterResourceBinding or ResourceBinding will continue to exist until the resource template is gone.
func (d *ResourceDetector) HandleClusterPropagationPolicyDeletion(policyName string) error {

@RainbowMango
Copy link
Member

I'm considering a case,

  • S0: Suppose we have a PropagationPolicy a which propagates a deployment to member1.
  • S1: Remove PropagationPolicy a;
  • S2: Create PropagationPolicy b which propagates the same deployment to member2.

Will the deployment be removed from member1? If not, does the behavior meets our expectation?

Signed-off-by: junqian <junqian@tencent.com>
@qianjun1993
Copy link
Contributor Author

I'm considering a case,

  • S0: Suppose we have a PropagationPolicy a which propagates a deployment to member1.
  • S1: Remove PropagationPolicy a;
  • S2: Create PropagationPolicy b which propagates the same deployment to member2.

Will the deployment be removed from member1? If not, does the behavior meets our expectation?

The deployment will be removed from member1

@RainbowMango
Copy link
Member

I found this issue twice today. Let's re-trigger it.

Installing 'kubectl v1.18.0' for you, may require the root privileges
curl: (56) TCP connection reset by peer
Error: Process completed with exit code 1.

@XiShanYongYe-Chang
Copy link
Member

/lgtm

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Aug 25, 2021
@RainbowMango RainbowMango changed the title stop remove ClusterResourceBindings labeled with policy keep ResourceBinding when removing PropagationPolicy Aug 25, 2021
@RainbowMango RainbowMango added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 25, 2021
@RainbowMango
Copy link
Member

/approve
Thanks.

@karmada-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Do not delete the work when PropagationPolicy is deleted
4 participants