Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crashed w/ runtime error [Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference] #1

Closed
galregev opened this issue Dec 13, 2023 · 5 comments · Fixed by #2
Closed

Crashed w/ runtime error [Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference] #1

galregev opened this issue Dec 13, 2023 · 5 comments · Fixed by #2
Assignees
@akyriako
Labels
bug Something isn't working

Comments

@galregev
Copy link

galregev commented Dec 13, 2023 •
edited
Loading

Hey there 👋🏻

I'm running on AWS EKS cluster v1.25.

The sleepcycles were installed on the cluster and the label were configured on one of the deployments.

Here are some logs from sleepcycles pod:

1.70249953358739e+09	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": "127.0.0.1:8080"}
1.7024995335878196e+09	INFO	setup	starting manager
1.7024995335883129e+09	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
1.7024995335883129e+09	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
I1213 20:32:13.588770       1 leaderelection.go:248] attempting to acquire leader lease sleepcycles-system/04cbe4c0.rekuberate.io...
I1213 20:32:30.242666       1 leaderelection.go:258] successfully acquired lease sleepcycles-system/04cbe4c0.rekuberate.io
1.702499550243052e+09	DEBUG	events	Normal	{"object": {"kind":"Lease","namespace":"sleepcycles-system","name":"04cbe4c0.rekuberate.io","uid":"294d077b-9dfd-4795-8120-99bddec3cd2b","apiVersion":"coordination.k8s.io/v1","resourceVersion":"852177480"}, "reason": "LeaderElection", "message": "sleepcycles-controller-manager-bf6c64857-j5k8d_808b4e88-ba76-4555-b558-8d7b48640b11 became leader"}
1.7024995502432828e+09	INFO	Starting EventSource	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "source": "kind source: *v1alpha1.SleepCycle"}
1.7024995502433164e+09	INFO	Starting Controller	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle"}
1.7024995503444824e+09	INFO	Starting workers	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "worker count": 1}
1.7024995505854793e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995508859751e+09	INFO	🕑 Processing CronJobs	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995509862914e+09	INFO	📦 Processing StatefulSets	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995510869236e+09	INFO	📈 Processing HorizontalPodAutoscalers	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995510988872e+09	INFO	🔁 Requeue	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown", "next-op": "Shutdown", "after": "28.901116474s"}
1.7024995511085997e+09	INFO	🔁 Requeue	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown", "next-op": "Shutdown", "after": "28.891404078s"}
1.7024995800032704e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995800034337e+09	INFO	🕑 Processing CronJobs	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995800034702e+09	INFO	📦 Processing StatefulSets	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.702499580003513e+09	INFO	📈 Processing HorizontalPodAutoscalers	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown"}
1.7024995800151165e+09	INFO	🔁 Requeue	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample1", "op": "Shutdown", "next-op": "Shutdown", "after": "59.984887885s"}
1.7024995800156264e+09	INFO	Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample1","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample1", "reconcileID": "f199f8db-368c-43e0-ad11-9c304278acdf"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x13881bb]

goroutine 201 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:118 +0x1f4
panic({0x14c2c80, 0x2328090})
	/usr/local/go/src/runtime/panic.go:838 +0x207
github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).getCurrentScheduledOperation(_, {{{0x13b128d, 0xa}, {0xc001af5780, 0x1b}}, {{0xc001af9b00, 0x12}, {0x0, 0x0}, {0xc001af9b30, ...}, ...}, ...})
	/workspace/controllers/sleepcycle_controller.go:455 +0x17b
github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile(0xc000490ab0, {0x18b67c0, 0xc001ba44e0}, {{{0xc0006803a8?, 0x10?}, {0xc000680390?, 0x40d787?}}})
	/workspace/controllers/sleepcycle_controller.go:84 +0x3a5
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x18b6718?, {0x18b67c0?, 0xc001ba44e0?}, {{{0xc0006803a8?, 0x15c5740?}, {0xc000680390?, 0x4041f4?}}})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121 +0xc8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0001a6b40, {0x18b6718, 0xc00057f4c0}, {0x1512b20?, 0xc0029332c0?})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320 +0x33c
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0001a6b40, {0x18b6718, 0xc00057f4c0})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273 +0x1d9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:230 +0x325

Here is the output from **k get sleepcycles.core.rekuberate.io -w**:

NAME                 SHUTDOWN      WAKEUP   ENABLED   NEXT SHUTDOWN          NEXT WAKE UP   LAST RUN               LAST RUN SUCCESS
sleepcycle-sample1   */1 * * * *            true      2023-12-13T20:28:00Z                  2023-12-13T20:27:00Z   true
sleepcycle-sample1   */1 * * * *            true      2023-12-13T20:33:00Z                  2023-12-13T20:32:31Z   true
sleepcycle-sample1   */1 * * * *            true      2023-12-13T20:34:00Z                  2023-12-13T20:33:00Z   true
sleepcycle-sample1   */1 * * * *            true      2023-12-13T20:39:00Z                  2023-12-13T20:38:30Z   true
sleepcycle-sample1   */1 * * * *            true      2023-12-13T20:40:00Z                  2023-12-13T20:39:00Z   true

🙏🏻 Thanks you in advance, Let me know in case there is some issue template.
@akyriako akyriako self-assigned this Dec 14, 2023
@akyriako
Copy link
Member

Hi @galregev, thanks for finding and opening the issue, I am investigating it.

@akyriako akyriako added the bug Something isn't working label Dec 14, 2023
akyriako added a commit that referenced this issue Dec 14, 2023
@akyriako
fix for issue #1, added event recorder
7ff7ba8
@akyriako akyriako linked a pull request Dec 14, 2023 that will close this issue
fix for issue #1, added event recorder #2
Merged
akyriako added a commit that referenced this issue Dec 14, 2023
@akyriako
Merge pull request #2 from rekuberate-io/1-crashed-w-runtime-error-ob…
969a6e1 
…served-a-panic-in-reconciler-runtime-error-invalid-memory-address-or-nil-pointer-dereference

fix for issue #1, added event recorder
@akyriako
Copy link
Member

Hi @galregev, the bug is corrected. Please try it again and let me know if you face any issues with that or anything else in the future.

@galregev
Copy link
Author

Hey @akyriako Thank you for responding.
I've pulled those changes in the git repo, and new Docker image were built, but it seems like the same result.

Here is the k get sleepcycles.core.rekuberate.io:

NAME                SHUTDOWN      WAKEUP        ENABLED   NEXT SHUTDOWN          NEXT WAKE UP           LAST RUN               LAST RUN SUCCESS
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:30:00Z   true
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:00Z
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:01Z
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:02Z
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:05Z
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:10Z
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:31:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:20Z
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:32:00Z   2023-12-14T08:32:00Z   2023-12-14T08:31:41Z   true
sleepcycle-sample   */1 * * * *   */2 * * * *   true      2023-12-14T08:33:00Z   2023-12-14T08:34:00Z   2023-12-14T08:32:00Z   true

And some logs:

sleepcycles-controller-manager-6996bdb869-5ncpc kube-rbac-proxy I1214 08:28:47.528021       1 main.go:186] Valid token audiences:
sleepcycles-controller-manager-6996bdb869-5ncpc kube-rbac-proxy I1214 08:28:47.528170       1 main.go:316] Generating self signed cert as no cert is provided
sleepcycles-controller-manager-6996bdb869-5ncpc kube-rbac-proxy I1214 08:28:49.719294       1 main.go:366] Starting TCP socket on 0.0.0.0:8443
sleepcycles-controller-manager-6996bdb869-5ncpc kube-rbac-proxy I1214 08:28:49.719735       1 main.go:373] Listening securely on 0.0.0.0:8443
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425346271825e+09	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": "127.0.0.1:8080"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425346276836e+09	INFO	setup	starting manager
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425346280844e+09	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425346280997e+09	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager I1214 08:28:54.628286       1 leaderelection.go:248] attempting to acquire leader lease sleepcycles-system/04cbe4c0.rekuberate.io...
sleepcycles-controller-manager-6996bdb869-5ncpc manager I1214 08:29:11.559607       1 leaderelection.go:258] successfully acquired lease sleepcycles-system/04cbe4c0.rekuberate.io
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425515596983e+09	DEBUG	events	Normal	{"object": {"kind":"Lease","namespace":"sleepcycles-system","name":"04cbe4c0.rekuberate.io","uid":"294d077b-9dfd-4795-8120-99bddec3cd2b","apiVersion":"coordination.k8s.io/v1","resourceVersion":"852993923"}, "reason": "LeaderElection", "message": "sleepcycles-controller-manager-6996bdb869-5ncpc_398eec6d-ef1c-468a-990b-2855748832da became leader"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425515601757e+09	INFO	Starting EventSource	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "source": "kind source: *v1alpha1.SleepCycle"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425515602224e+09	INFO	Starting Controller	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425516607475e+09	INFO	Starting workers	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "worker count": 1}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025425516710978e+09	INFO	🔁 Requeue	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Watch", "next-op": "Wakeup", "after": "48.32890587s"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426002022455e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Wakeup"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542600403111e+09	INFO	🕑 Processing CronJobs	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Wakeup"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426005036705e+09	INFO	📦 Processing StatefulSets	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Wakeup"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542600604469e+09	INFO	📈 Processing HorizontalPodAutoscalers	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Wakeup"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426006151695e+09	INFO	🔁 Requeue	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Wakeup", "next-op": "Shutdown", "after": "59.38483418s"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660002135e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660002183e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660023864e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600247126e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852994871"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.029826       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852994871", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), Count:1, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot create resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600377748e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "80561b4a-ed1d-49db-bf0e-7ba6b7d01172", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600438006e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600438964e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600467587e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600473592e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.055351       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 46925823, time.Local), Count:2, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600605037e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "1ca069e5-4bce-444c-b3ce-cbca97cae708", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660071149e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600715213e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660075206e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600756118e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.078586       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 75334939, time.Local), Count:3, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426600865963e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "bc3f2cd6-eb82-4b41-b01a-951e7abb4e7e", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601072092e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601072624e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.70254266011041e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601111221e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.118032       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 110500573, time.Local), Count:4, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660123124e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "e035a394-1601-469e-8151-6e19f8037796", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601643617e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601645124e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601678796e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660168949e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.171714       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 167959415, time.Local), Count:5, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426601775076e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "4a4b4dce-84f7-47c3-8c73-5b6394621d03", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426602584732e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426602585168e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660261376e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426602619765e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.266263       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 261524206, time.Local), Count:6, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426602700946e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "0cd76f04-14fd-467f-aa81-5a47b0e666f4", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426604313436e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426604314775e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426604339128e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660434549e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.441561       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 434266489, time.Local), Count:7, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426604524758e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "33a4e848-f5ae-4150-8183-4b0ed68ed631", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542660773494e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426607735436e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426607770467e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426607791965e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:00.781373       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 777187980, time.Local), Count:8, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426607868295e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "3a95a8e3-270d-4445-9e98-abff211fb36b", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426614279728e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542661428002e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.70254266143072e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426614309454e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995940"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:01.435141       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995940", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 1, 430802538, time.Local), Count:9, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542661440872e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "f645123b-4987-412f-92d2-80e69967f7ef", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426627217631e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426627217915e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426627239087e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426627243154e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995962"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:02.727021       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995962", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 2, 723995344, time.Local), Count:10, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426627358105e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "6d9e5ae2-fee8-4d31-b3d1-6fb592e95ee2", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426652969096e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426652971797e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426652998998e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426653011668e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852995977"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:05.310430       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852995977", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 5, 299983968, time.Local), Count:11, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542665314612e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "25550dc0-61df-4477-8c11-fa53fa6fd91b", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426704351192e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426704351406e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426704381435e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426704384148e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852996020"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:10.442303       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852996020", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 10, 438206581, time.Local), Count:12, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426704526584e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "67116e8e-f507-494c-8ca8-cb38c11ee246", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426806933813e+09	INFO	📚 Processing Deployments	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542680693425e+09	INFO	🌙 Scale Down Deployment	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "targetReplicas": 0}
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542680696065e+09	ERROR	🛑️ Scaling Deployment failed	{"namespace": "es-digital-journeys", "sleepcycle": "sleepcycle-sample", "op": "Shutdown", "deployment": "es-digital-journeys/es-digital-journeys-player-celery", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager github.com/rekuberate-io/sleepcycles/controllers.(*SleepCycleReconciler).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/workspace/controllers/sleepcycle_controller.go:129
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.702542680696387e+09	DEBUG	events	Warning	{"object": {"kind":"SleepCycle","namespace":"es-digital-journeys","name":"sleepcycle-sample","uid":"547a4395-6a6c-446f-ab2c-4e376b3e921f","apiVersion":"core.rekuberate.io/v1alpha1","resourceVersion":"852996113"}, "reason": "SleepCycleOpFailure", "message": "shutdown on es-digital-journeys/es-digital-journeys-player-celery failed"}
sleepcycles-controller-manager-6996bdb869-5ncpc manager E1214 08:31:20.700324       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"sleepcycle-sample.17a0a58761afed90", GenerateName:"", Namespace:"es-digital-journeys", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"SleepCycle", Namespace:"es-digital-journeys", Name:"sleepcycle-sample", UID:"547a4395-6a6c-446f-ab2c-4e376b3e921f", APIVersion:"core.rekuberate.io/v1alpha1", ResourceVersion:"852996113", FieldPath:""}, Reason:"SleepCycleOpFailure", Message:"shutdown on es-digital-journeys/es-digital-journeys-player-celery failed", Source:v1.EventSource{Component:"rekuberate-io/sleepcycles", Host:""}, FirstTimestamp:time.Date(2023, time.December, 14, 8, 31, 0, 24135056, time.Local), LastTimestamp:time.Date(2023, time.December, 14, 8, 31, 20, 696221051, time.Local), Count:13, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "sleepcycle-sample.17a0a58761afed90" is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot patch resource "events" in API group "" in the namespace "es-digital-journeys"' (will not retry!)
sleepcycles-controller-manager-6996bdb869-5ncpc manager 1.7025426807087038e+09	ERROR	Reconciler error	{"controller": "sleepcycle", "controllerGroup": "core.rekuberate.io", "controllerKind": "SleepCycle", "sleepCycle": {"name":"sleepcycle-sample","namespace":"es-digital-journeys"}, "namespace": "es-digital-journeys", "name": "sleepcycle-sample", "reconcileID": "7e78c425-d0d9-49cc-9811-fe2d2b27ada5", "error": "deployments.apps \"es-digital-journeys-player-celery\" is forbidden: User \"system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager\" cannot update resource \"deployments\" in API group \"apps\" in the namespace \"es-digital-journeys\""}
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273
sleepcycles-controller-manager-6996bdb869-5ncpc manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sleepcycles-controller-manager-6996bdb869-5ncpc manager 	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234

@akyriako
Copy link
Member

akyriako commented Dec 14, 2023 •
edited
Loading

Hi @galregev, I am sorry you don't have a smooth experience. In the new log that you sent I don't see the old error anymore (it is fixed), but I see something totally different:

is forbidden: User "system:serviceaccount:sleepcycles-system:sleepcycles-controller-manager" cannot update resource "deployments" in API group "apps" in the namespace "es-digital-journeys""}

That is an RBAC issue.

@akyriako akyriako reopened this Dec 14, 2023
akyriako added a commit that referenced this issue Dec 14, 2023
@akyriako
fix for issue #1, added RBACs missing
4f33122
akyriako added a commit that referenced this issue Dec 14, 2023
@akyriako
Merge pull request #3 from rekuberate-io/1-crashed-w-runtime-error-ob…
7ede16c 
…served-a-panic-in-reconciler-runtime-error-invalid-memory-address-or-nil-pointer-dereference

fix for issue #1, added RBACs missing
akyriako added a commit that referenced this issue Dec 14, 2023
@akyriako
fix for issue #1, time encoder
b35d504
akyriako added a commit that referenced this issue Dec 14, 2023
@akyriako
Merge pull request #4 from rekuberate-io/1-crashed-w-runtime-error-ob…
55d7304 
…served-a-panic-in-reconciler-runtime-error-invalid-memory-address-or-nil-pointer-dereference

fix for issue #1, time encoder
@akyriako
Copy link
Member

Fixed

@akyriako akyriako moved this to Done in sleepcycles Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@akyriako akyriako

Labels
bug Something isn't working
Projects
sleepcycles
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix for issue #1, added event recorder rekuberate-io/sleepcycles
2 participants
@akyriako @galregev