[Bug] Report generation failure

[andrewazores]

Current Behavior

On latest main smoketest, try to generate a report by expanding a recording table row (active or archived).

cryostat_1        | ERROR: HTTP Request to /api/v3/reports/OGFsSGhsMmQ2ZE5EN0FaWHpScGVvYXNsaFExQ3BaczFXVlNuT0k3RTU5VT0vY29tcG9zZS1qZnItZGF0YXNvdXJjZS0xX3NuYXBzaG90LTJfMjAyNDA0MTNUMDM0MzQ1Wi5qZnI failed, error id: 5f198f2d-6c51-4197-8426-7fdda8d25829-2
cryostat_1        | java.lang.IllegalStateException: Recursive update
cryostat_1        | 	at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1991)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2675)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2658)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.LocalAsyncCache$AsyncAsMapView.computeIfAbsent(LocalAsyncCache.java:365)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.LocalAsyncCache$AsyncAsMapView.computeIfAbsent(LocalAsyncCache.java:293)
cryostat_1        | 	at io.quarkus.cache.runtime.caffeine.CaffeineCacheImpl$3.get(CaffeineCacheImpl.java:109)
cryostat_1        | 	at io.quarkus.cache.runtime.caffeine.CaffeineCacheImpl$3.get(CaffeineCacheImpl.java:103)
cryostat_1        | 	at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
cryostat_1        | 	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:24)
cryostat_1        | 	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
cryostat_1        | 	at io.smallrye.mutiny.operators.uni.UniOnItemTransform.subscribe(UniOnItemTransform.java:22)
cryostat_1        | 	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
cryostat_1        | 	at io.smallrye.mutiny.operators.uni.UniOnItemTransform.subscribe(UniOnItemTransform.java:22)
cryostat_1        | 	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:51)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.with(UniSubscribe.java:110)
cryostat_1        | 	at io.smallrye.mutiny.operators.uni.UniSubscribeToCompletionStage.subscribe(UniSubscribeToCompletionStage.java:30)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.asCompletionStage(UniSubscribe.java:174)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.asCompletionStage(UniSubscribe.java:162)
cryostat_1        | 	at io.smallrye.mutiny.Uni.subscribeAsCompletionStage(Uni.java:141)
cryostat_1        | 	at io.quarkus.cache.runtime.caffeine.CaffeineCacheImpl$3$1.apply(CaffeineCacheImpl.java:116)
cryostat_1        | 	at io.quarkus.cache.runtime.caffeine.CaffeineCacheImpl$3$1.apply(CaffeineCacheImpl.java:110)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.LocalAsyncCache$AsyncAsMapView.lambda$computeIfAbsent$0(LocalAsyncCache.java:366)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2677)
cryostat_1        | 	at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2675)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2658)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.LocalAsyncCache$AsyncAsMapView.computeIfAbsent(LocalAsyncCache.java:365)
cryostat_1        | 	at com.github.benmanes.caffeine.cache.LocalAsyncCache$AsyncAsMapView.computeIfAbsent(LocalAsyncCache.java:293)
cryostat_1        | 	at io.quarkus.cache.runtime.caffeine.CaffeineCacheImpl$3.get(CaffeineCacheImpl.java:109)
cryostat_1        | 	at io.quarkus.cache.runtime.caffeine.CaffeineCacheImpl$3.get(CaffeineCacheImpl.java:103)
cryostat_1        | 	at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
cryostat_1        | 	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:24)
cryostat_1        | 	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
cryostat_1        | 	at io.smallrye.mutiny.operators.uni.UniOnItemTransform.subscribe(UniOnItemTransform.java:22)
cryostat_1        | 	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:51)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.with(UniSubscribe.java:110)
cryostat_1        | 	at io.smallrye.mutiny.groups.UniSubscribe.with(UniSubscribe.java:88)
cryostat_1        | 	at org.jboss.resteasy.reactive.server.handlers.UniResponseHandler.handle(UniResponseHandler.java:19)
cryostat_1        | 	at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:150)
cryostat_1        | 	at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:145)
cryostat_1        | 	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:576)
cryostat_1        | 	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
cryostat_1        | 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
cryostat_1        | 	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
cryostat_1        | 	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
cryostat_1        | 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
cryostat_1        | 	at java.base/java.lang.Thread.run(Thread.java:840)

Expected Behavior

No response

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

[andrewazores]

Bisection shows that the first bad commit was #352 , interestingly enough.

[andrewazores]

Generation works with Quarkus 3.2.6 and 3.2.7, but fails with 3.2.8-3.2.11. Nothing in the 3.2.8 release notes jumps out as an obvious cause.

[andrewazores]

Could it be that the layered caching is no longer layering as expected and the memory cache layer is getting an instance of itself, instead of the storage layer to delegate to?

[andrewazores]

Setting QUARKUS_CACHE_ENABLED: "false" in smoketest cryostat.yml fixes the apparent bug, so it does look like a memory caching layer problem.

[andrewazores]

My closest finding so far is quarkusio/quarkus#35664 / quarkusio/quarkus#35674 . If I remove the default methods from the interface definition and move those directly into the concrete implementations then the bug is solved.