feat: add database spec to CRDs (#136) #343

kosmoz · 2023-09-27T15:14:22Z

Also fixes #221

pmig

I tried to perform a MariaDb storage increase and received following erorr:

2023-10-04T09:57:42.777+02:00 [pool-6-thread-5] INFO  e.g.o.a.m.d.MatomoMariaDB - storage request for Matomo{matomo.matomo} has changed (actual: 10Gi, desired: 11Gi). Now updating the PersistentVolumeClaim and recreating MariaDB 
2023-10-04T09:57:44.097+02:00 [pool-6-thread-5] INFO  e.g.o.a.m.d.MatomoMariaDB - MariaDB for Matomo{matomo.matomo} recreated 
2023-10-04T09:57:44.132+02:00 [-1293967722-pool-2-thread-15] ERROR i.f.k.c.i.i.c.Reflector - watch failed for mariadb.mmontes.io/v1alpha1/mariadbs, will stop 
io.fabric8.kubernetes.client.WatcherException: Unexpected exception processing watch event
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:384)
	at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:52)
	at io.fabric8.kubernetes.client.okhttp.OkHttpWebSocketImpl$1.onMessage(OkHttpWebSocketImpl.java:116)
	at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.kt:333)
	at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.kt:245)
	at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.kt:106)
	at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.kt:293)
	at okhttp3.internal.ws.RealWebSocket$connect$1.onResponse(RealWebSocket.kt:195)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalArgumentException: Instantiation of [simple type, class eu.glasskube.operator.infra.mariadb.MariaDBStatus] value failed for JSON property conditions due to missing (therefore NULL) value for creator parameter conditions which is a non-nullable type
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: eu.glasskube.operator.infra.mariadb.MariaDB["status"]->eu.glasskube.operator.infra.mariadb.MariaDBStatus["conditions"])
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4544)
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4475)
	at io.fabric8.kubernetes.client.utils.KubernetesSerialization.convertValue(KubernetesSerialization.java:368)
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.eventReceived(AbstractWatchManager.java:281)
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:371)
	... 11 common frames omitted
Caused by: com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class eu.glasskube.operator.infra.mariadb.MariaDBStatus] value failed for JSON property conditions due to missing (therefore NULL) value for creator parameter conditions which is a non-nullable type
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: eu.glasskube.operator.infra.mariadb.MariaDB["status"]->eu.glasskube.operator.infra.mariadb.MariaDBStatus["conditions"])
	at com.fasterxml.jackson.module.kotlin.KotlinValueInstantiator.createFromObjectWith(KotlinValueInstantiator.kt:84)
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:525)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at io.fabric8.kubernetes.model.jackson.SettableBeanPropertyDelegate.deserializeAndSet(SettableBeanPropertyDelegate.java:134)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4539)
	... 15 common frames omitted
2023-10-04T09:57:44.136+02:00 [-1293967722-pool-2-thread-15] ERROR i.j.o.a.c.ConfigurationService - Fatal error in informer: mariadb.mmontes.io/v1alpha1/mariadbs. Stopping the operator 
io.fabric8.kubernetes.client.WatcherException: Unexpected exception processing watch event
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:384)
	at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:52)
	at io.fabric8.kubernetes.client.okhttp.OkHttpWebSocketImpl$1.onMessage(OkHttpWebSocketImpl.java:116)
	at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.kt:333)
	at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.kt:245)
	at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.kt:106)
	at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.kt:293)
	at okhttp3.internal.ws.RealWebSocket$connect$1.onResponse(RealWebSocket.kt:195)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalArgumentException: Instantiation of [simple type, class eu.glasskube.operator.infra.mariadb.MariaDBStatus] value failed for JSON property conditions due to missing (therefore NULL) value for creator parameter conditions which is a non-nullable type
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: eu.glasskube.operator.infra.mariadb.MariaDB["status"]->eu.glasskube.operator.infra.mariadb.MariaDBStatus["conditions"])
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4544)
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4475)
	at io.fabric8.kubernetes.client.utils.KubernetesSerialization.convertValue(KubernetesSerialization.java:368)
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.eventReceived(AbstractWatchManager.java:281)
	at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:371)
	... 11 common frames omitted
Caused by: com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class eu.glasskube.operator.infra.mariadb.MariaDBStatus] value failed for JSON property conditions due to missing (therefore NULL) value for creator parameter conditions which is a non-nullable type
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: eu.glasskube.operator.infra.mariadb.MariaDB["status"]->eu.glasskube.operator.infra.mariadb.MariaDBStatus["conditions"])
	at com.fasterxml.jackson.module.kotlin.KotlinValueInstantiator.createFromObjectWith(KotlinValueInstantiator.kt:84)
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:525)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1409)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at io.fabric8.kubernetes.model.jackson.SettableBeanPropertyDelegate.deserializeAndSet(SettableBeanPropertyDelegate.java:134)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4539)
	... 15 common frames omitted
2023-10-04T09:57:44.141+02:00 [SpringApplicationShutdownHook] INFO  i.j.o.Operator - Operator SDK 4.4.4 is shutting down... 
2023-10-04T09:57:44.194+02:00 [ReconcilerExecutor-matomoreconciler-111] ERROR i.j.o.p.d.w.WorkflowReconcileExecutor - Thread interrupted for primary: ResourceID{name='matomo', namespace='matomo'} 
java.lang.InterruptedException: null
	at java.base/java.lang.Object.wait(Native Method)
	at java.base/java.lang.Object.wait(Object.java:338)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.AbstractWorkflowExecutor.waitForScheduledExecutionsToRun(AbstractWorkflowExecutor.java:49)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowReconcileExecutor.reconcile(WorkflowReconcileExecutor.java:44)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.DefaultWorkflow.reconcile(DefaultWorkflow.java:93)
	at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:147)
	at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:110)
	at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219)
	at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:109)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:140)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:121)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:91)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64)
	at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:417)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2023-10-04T09:57:44.195+02:00 [ReconcilerExecutor-matomoreconciler-111] ERROR i.j.o.p.e.ReconciliationDispatcher - Error during event processing ExecutionScope{ resource id: ResourceID{name='matomo', namespace='matomo'}, version: 664835420} failed. 
io.javaoperatorsdk.operator.OperatorException: java.lang.InterruptedException
	at io.javaoperatorsdk.operator.processing.dependent.workflow.AbstractWorkflowExecutor.waitForScheduledExecutionsToRun(AbstractWorkflowExecutor.java:61)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.WorkflowReconcileExecutor.reconcile(WorkflowReconcileExecutor.java:44)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.DefaultWorkflow.reconcile(DefaultWorkflow.java:93)
	at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:147)
	at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:110)
	at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219)
	at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:109)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:140)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:121)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:91)
	at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64)
	at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:417)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException: null
	at java.base/java.lang.Object.wait(Native Method)
	at java.base/java.lang.Object.wait(Object.java:338)
	at io.javaoperatorsdk.operator.processing.dependent.workflow.AbstractWorkflowExecutor.waitForScheduledExecutionsToRun(AbstractWorkflowExecutor.java:49)
	... 14 common frames omitted

I am already on the next version on the MariaDB Operator
mariadb-operator-0.20.0 v0.0.20.

I did not test it with 0.19, but maybe it makes sense to already support the next version?
-> if not LGTM

Volume expension for Postgres worked fine!

kosmoz · 2023-10-04T08:35:42Z

OK I just checked and I realised, I was testing with the MariaDB operator on v0.15.0! For sure there were some breaking changes in the meantime, so I will try again with the latest version.

Judging by the logs excerpt you posted though, I believe that the volume expansion itself actually worked, however there is a JSON parsing error afterwards.

kosmoz · 2023-10-04T09:34:28Z

I just checked and the error also occurs in main, so I'm going to create another PR for this.

…pec (#136)

kosmoz · 2023-10-04T10:13:59Z

I created a patch for this bug in another PR. After rebasing this branch, volume expansion for MariaDBs works with version 0.20.0 of the MariaDB operator.

kosmoz self-assigned this Sep 27, 2023

kosmoz linked an issue Sep 27, 2023 that may be closed by this pull request

[Feature]: Custom backup location #136

Closed

kosmoz requested a review from pmig September 28, 2023 13:04

kosmoz changed the title ~~feat: add optional backup spec to CRDs (#136)~~ feat: add database spec to CRDs (#136) Sep 28, 2023

kosmoz force-pushed the 136-feature-custom-backup-location branch 3 times, most recently from 72d4236 to ce85377 Compare October 3, 2023 13:10

kosmoz requested review from pmig and removed request for pmig October 3, 2023 13:10

pmig reviewed Oct 4, 2023

View reviewed changes

kosmoz mentioned this pull request Oct 4, 2023

[BUG]: JSONParseException when using MariaDB Operator v0.20.0 #354

Closed

2 tasks

kosmoz added 9 commits October 4, 2023 11:48

feat: add optional backup spec to CRDs (#136)

1647a2d

refactor: change database backup spec to be subresource of database s…

4b2d623

…pec (#136)

feat: add instances and storage spec to postgres configuration (#221)

380adfb

fix: add precondition for backup minio buckets (#136)

35c810a

feat(Matomo): add database storage spec (#136)

a2c4fa9

chore: update generated crds (#136)

bd2151e

chore: change cluster-role.yaml to unified formatting

c6daba3

chore: add permission to delete mariadbs to cluster-role.yaml (#136)

369d6a6

chore: update intellij settings

dd3a0f2

kosmoz force-pushed the 136-feature-custom-backup-location branch from ce85377 to dd3a0f2 Compare October 4, 2023 10:02

kosmoz merged commit ccadbca into main Oct 4, 2023
3 checks passed

kosmoz deleted the 136-feature-custom-backup-location branch October 4, 2023 10:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add database spec to CRDs (#136) #343

feat: add database spec to CRDs (#136) #343

kosmoz commented Sep 27, 2023 •

edited

Loading

pmig left a comment

kosmoz commented Oct 4, 2023

kosmoz commented Oct 4, 2023

kosmoz commented Oct 4, 2023

feat: add database spec to CRDs (#136) #343

feat: add database spec to CRDs (#136) #343

Conversation

kosmoz commented Sep 27, 2023 • edited Loading

pmig left a comment

Choose a reason for hiding this comment

kosmoz commented Oct 4, 2023

kosmoz commented Oct 4, 2023

kosmoz commented Oct 4, 2023

kosmoz commented Sep 27, 2023 •

edited

Loading