-
Notifications
You must be signed in to change notification settings - Fork 188
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bug] HelmRepository blocked if secret on startup not exists #1173
Comments
Does it happen with the ErrorHandling here? source-controller/internal/controller/helmrepository_controller.go Lines 411 to 416 in 7f40be7
On the GitRepository (where it works), there we god an "Generic" Error: source-controller/internal/controller/gitrepository_controller.go Lines 485 to 491 in 7f40be7
maybe the old error lead them to block permenently |
Hi, I just tried it but I couldn't reproduce it. apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
name: podinfo
namespace: default
spec:
interval: 1m
url: https://stefanprodan.github.io/podinfo
secretRef:
name: "example" The secret doesn't exist yet.
The status of helmrepo shows ( status:
conditions:
- lastTransitionTime: "2023-07-21T10:34:45Z"
message: building artifact
observedGeneration: 1
reason: ProgressingWithRetry
status: "True"
type: Reconciling
- lastTransitionTime: "2023-07-21T10:34:45Z"
message: 'failed to get secret ''default/example'': secrets "example" not found'
observedGeneration: 1
reason: AuthenticationFailed
status: "False"
type: Ready
- lastTransitionTime: "2023-07-21T10:34:40Z"
message: 'failed to get secret ''default/example'': secrets "example" not found'
observedGeneration: 1
reason: AuthenticationFailed
status: "True"
type: FetchFailed
observedGeneration: -1 After creating the secret, within a few seconds, the logs show
and the helmrepo status shows: status:
artifact:
digest: sha256:80b091a3a69b9ecfebde40ce2a5f19e95f8f8ea956bd5635a31701f7fad1616e
lastUpdateTime: "2023-07-21T10:36:16Z"
path: helmrepository/default/podinfo/index-80b091a3a69b9ecfebde40ce2a5f19e95f8f8ea956bd5635a31701f7fad1616e.yaml
revision: sha256:80b091a3a69b9ecfebde40ce2a5f19e95f8f8ea956bd5635a31701f7fad1616e
size: 43126
url: http://:0/helmrepository/default/podinfo/index-80b091a3a69b9ecfebde40ce2a5f19e95f8f8ea956bd5635a31701f7fad1616e.yaml
conditions:
- lastTransitionTime: "2023-07-21T10:36:16Z"
message: 'stored artifact: revision ''sha256:80b091a3a69b9ecfebde40ce2a5f19e95f8f8ea956bd5635a31701f7fad1616e'''
observedGeneration: 1
reason: Succeeded
status: "True"
type: Ready
- lastTransitionTime: "2023-07-21T10:36:16Z"
message: 'stored artifact: revision ''sha256:80b091a3a69b9ecfebde40ce2a5f19e95f8f8ea956bd5635a31701f7fad1616e'''
observedGeneration: 1
reason: Succeeded
status: "True"
type: ArtifactInStorage
observedGeneration: 1
... An object can get blocked if they have a |
@genofire when reporting bugs please say which version you're using by simply posting the |
|
That's the CLI version, what about controllers and CRDs? |
no i mean, that the namespace has the version-label of 2.0.0-rc5 - have edit / update the message |
Can you please upgrade to Flux v2.0.1 and see if this issue persists? |
That needs time -> we have 30 clusters with staging |
Not asking you to upgrade all of them, just one to rerun the test. We've tried to replicate this with 2.0.1 and the HelmRepository is not getting stuck. Also what type of repo are you using? OCI or Helm HTTP? |
It wold also be helpful if you can post here |
so secret exists for 31 minutes, now: helmrepo:
oci helmrepo:
|
If you run |
if i trigger it twice:
|
This is really strange, is your Kubernetes API under heavy load, is etcd having any issue? This may be a caching issue, we have disabled the caching of Secrets in our controllers but the API does it as well. |
It is your cloud provider IONOS ... we have no control over the etcd. my problem ist, i do not see any logs above a reconcileing of this helmrepository (other i see) ... like it is in stall. we have that problem daily over two month (always if we create a new cluster and install there your default resources) |
if you are right, that the kube-api request is under heavy load, so maybe we should timeout request there (maybe that is the problem), |
Steps:
Error Behavour:
Expected Behavour:
Workaround:
in fluxcd, version:
The text was updated successfully, but these errors were encountered: