Fix cache miss hit on resource recreation #1813
Conversation
kubevirt-bot
added
release-note
Pull Request Test Coverage Report for Build 1979952322
💛 - Coveralls |
|
/retest |
|
hco-e2e-image-index-sno-azure lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-sno-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-image-index-azure, hco-e2e-image-index-aws lanes succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-gcp In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-upgrade-prev-index-aws lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-prev-index-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/retest |
2 similar comments
|
/retest |
|
/retest |
|
@tiraboschi |
|
/retest |
1 similar comment
|
/retest |
tiraboschi
force-pushed
the
fix_cache_miss_hit
branch
from
3f013c9
to
d5a6325
Compare
|
hco-e2e-image-index-gcp, hco-e2e-image-index-azure lanes succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-aws, ci/prow/okd-hco-e2e-image-index-gcp In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-upgrade-prev-index-sno-azure lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-prev-index-sno-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/rebase |
hco-bot
force-pushed
the
fix_cache_miss_hit
branch
from
d5a6325
to
d5a290e
Compare
|
hco-e2e-upgrade-prev-index-aws lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-sno-azure, ci/prow/hco-e2e-upgrade-index-azure, ci/prow/hco-e2e-upgrade-prev-index-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-image-index-gcp, hco-e2e-image-index-aws lanes succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/retest |
|
ignoring slightly decreased coverage due to a path that cannot be covered with the fake client |
|
@tiraboschi: Overrode contexts on behalf of tiraboschi: coverage/coveralls In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
pkg/controller/operands/operand.go
Outdated
| cfg, configerr := config.GetConfig() | ||
| if configerr != nil { | ||
| req.Logger.Error(configerr, "failed creating a config for a custom client") | ||
| return &EnsureResult{ | ||
| Err: configerr, | ||
| } | ||
| } | ||
| apiClient, acerr := client.New(cfg, client.Options{ | ||
| Scheme: h.Scheme, | ||
| }) | ||
| if acerr != nil { | ||
| req.Logger.Error(acerr, "failed creating a custom client to bypass the cache") | ||
| return &EnsureResult{ | ||
| Err: acerr, | ||
| } | ||
| } | ||
| geterr := apiClient.Get(req.Ctx, key, found) | ||
| if geterr != nil { | ||
| req.Logger.Error(geterr, "failed trying to get the object bypassing the cache") | ||
| return &EnsureResult{ | ||
| Err: geterr, | ||
| } | ||
| } | ||
| originalClient := h.Client | ||
| h.Client = apiClient | ||
| existingcrresult := h.handleExistingCr(req, key, found, cr, res) | ||
| h.Client = originalClient | ||
| return existingcrresult | ||
| } |
There was a problem hiding this comment.
- Can you put all of this in a new function? the original is too long now.
- I wonder (not urgent) if we can change it to be "thread safe" if we'll want to call the operand ensure methods in their own goroutine. WDYT?
The k8s client is configured to use selectors based on label to watch only a subset of certain resources to limit the memory consumption. IF the user explictly removes that label from one of the watched object, the client cache will miss it and so the operator will try recreating it failing then with AlreadyExists. Let's explictly detect this corner case and fix it setting the missing label with a custom client to bypass the cache. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=2032837 Signed-off-by: Simone Tiraboschi <stirabos@redhat.com>
tiraboschi
force-pushed
the
fix_cache_miss_hit
branch
from
d5a290e
to
751f369
Compare
|
Kudos, SonarCloud Quality Gate passed!
|
|
ignoring slightly decreased coverage due to a path that cannot be covered with the fake client |
|
@tiraboschi: Overrode contexts on behalf of tiraboschi: coverage/coveralls In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-upgrade-index-sno-azure lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-index-sno-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/retest |
|
@tiraboschi: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
okd-hco-e2e-upgrade-index-gcp lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/okd-hco-e2e-upgrade-index-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: nunnatsa The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
kubevirt-bot
added
the
approved
|
/cherry-pick release-1.6 |
|
@tiraboschi: once the present PR merges, I will cherry-pick it on top of release-1.6 in a new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@tiraboschi: new pull request created: #1818 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
The k8s client is configured to use selectors
based on label to watch only a subset of
certain resources to limit the memory consumption.
IF the user explictly removes that label
from one of the watched object, the client
cache will miss it and so the operator will try
recreating it failing then with AlreadyExists.
Let's explictly detect this corner case and
fix it setting the missing label with
a custom client to bypass the cache.
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=2032837
Signed-off-by: Simone Tiraboschi stirabos@redhat.com
Reviewer Checklist
Release note: