CustomResourceDefinitions: withResourceVersion() causes NoSuchMethodError

[ljnelson]

I'll work over the next few days to get a reproducer, but I'm following the general example code in CRDExample.java.

I'm using kubernetes-client 3.2.0 and kubernetes-model 2.1.1.

I have a custom resource definition and a CustomResource of type Foo. I have a MixedOperation<Foo, FooList, DoneableFoo, Resource<Foo, DoneableFoo>> instance in my hand that resulted from invoking client.customResources(fooCrd, Foo.class, FooList.class, DoneableFoo.class).

Now instead of calling list() right away on it, I call withResourceVersion(0) in my particular use case (which for whatever it's worth is building out the Java equivalent of the Kubernetes-native tools/cache package). (Note that the Go code does the equivalent.)

With non-custom resources this withResourceVersion(0) invocation followed by a .list() invocation works just fine.

With custom resources, though, I see this (edited) stack:

io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62)
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withResourceVersion(BaseOperation.java:685)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withResourceVersion(BaseOperation.java:49)
[snip]
Caused by: java.lang.NoSuchMethodException: io.fabric8.kubernetes.client.dsl.internal.CustomResourceOperationsImpl.<init>(okhttp3.OkHttpClient, io.fabric8.kubernetes.client.Config, java.lang.String, java.lang.String, java.lang.String, java.lang.Boolean, com.foobar.Foo, java.lang.String, java.lang.Boolean, long, java.util.Map, java.util.Map, java.util.Map, java.util.Map, java.util.Map)
	at java.lang.Class.getConstructor0(Class.java:3082)
	at java.lang.Class.getConstructor(Class.java:1825)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withResourceVersion(BaseOperation.java:682)

The problem is at line 682 in BaseOperation.java. As you can clearly see, there is no such constructor in CustomResourceOperationsImpl.java.

[ljnelson]

FYI I have worked around this issue using a series of reflection hacks.

[ljnelson]

My workaround leads to a strange interaction with KubernetesDeserializer, where another problem lies.

Briefly, my workaround uses various horrible reflection hacks to call the actually existing constructor in CustomResourceOperationsImpl, supplying arguments that would be supplied were the constructor that the code is trying to call to exist.

In so doing, I see that the registerCustomKind method of KubernetesDeserializer is called. It is called without an apiVersion, so in the case of, for example, dev.foobar.com/v1#Foo, the custom Foo type is registered under simply Foo. I don't know whether this is deliberate or not, or whether this is a bug or not, or what the reasoning is (suppose my CRD is v2…?).

Then, in the bowels of KubernetesDeserializer, getTypeForKey will, when passed Foo, return the right class.

But sometimes getTypeForKey is passed not just Foo, but dev.foobar.com/v1#Foo (I see it when a WatchEvent containing a Foo is being deserialized). The getTypeForKey method seems to be expecting this to a certain degree, as it checks for the presence of a # and branches to additional logic if one is found in the incoming key. Thus, if it is handed dev.foobar.com/v1#Foo, surgery will be performed, and the key, temporarily, will be Foo.

Unfortunately then this key is prefixed with various kubernetes-client package names and sought again. In the case of certain custom resource operations, getTypeForKey thus yields null. In my case, the Foo type doesn't have one of these magic package names as a prefix in its class name. (Neither, I should point out, does Dummy.java in kubernetes-client-example.)

I wonder: is there a particular reason that CustomResourceOperationsImpl.java takes great care to call the variant of the registerCustomKind method without the api version? I mean, there is an apiVersion in play at that moment and everything—why not pass it here?

[xguerin]

@ljnelson I hit the same problem as you do. Custom kinds get registered without API version and are basically unusable. My initial test was successful because I happened to use the kind "Job", which I guess exists in other packages.

[ljnelson]

@xguerin You may wish to look at how I worked around the issue.

[xguerin]

Interestingly I am using your Controller ;) I’ll do some more digging and look at your workaround, but I half expected to be using it already.

[ljnelson]

@xguerin Ah; then yes, the workaround for this issue I list above is already in play. You'll also need to do something like this suitably early:

// Yet another hack
io.fabric8.kubernetes.internal.KubernetesDeserializer.registerCustomKind(YOUR_CRD_GROUP + "/" + YOUR_CRD_VERSION + "#Foo", Foo.class);

[rohanKanojia]

@ljnelson @xguerin : is this still an issue? @mgorniew raised a PR for this earlier which got merged.

[xguerin]

Excellent question. I'll update to a release post aug 28th and let you know.

[xguerin]

I am still on this. I am synchronizing with @ljnelson to check if both of our use cases got fixed.

[ljnelson]

Sorry for the delay; crawling back after the holidays!

[xguerin]

@rohanKanojia I was able to test 4.1.1 and unfortunately the problem does not seem to be solved. Here is the kind of error I am getting:

2019-02-15 22:12:33 ERROR WatchHTTPManager:293 - Could not deserialize watch event: {"apiVersion":"v1","kind":"List", "items": [{"kind":"Export","apiVersion":"my.controller.com/v1alpha1","metadata":{"annotations":{"my.controller.com/applicationName":"apps.pubsub::Exporter","my.controller.com/applicationScope":"Default"},"labels":{"app":"myapp","job":"test"},"name":"test-0-2","namespace":"test","ownerReferences":[{"apiVersion":"my.controller.com/v1alpha1","kind":"Job","blockOwnerDeletion":true,"controller":true,"name":"test"}]},"spec":{"peId":0,"portId":2,"stream":{"properties":null,"exportOperName":"ExportOp2","name":"hcSectorBargains","allowFilter":true,"congestionPolicy":"WAIT"}}}], "metadata": {"resourceVersion":"v1alpha1"}}
com.fasterxml.jackson.databind.JsonMappingException: No resource type found for:my.controller.com/v1alpha1#Export
 at [Source: N/A; line: -1, column: -1] (through reference chain: io.fabric8.kubernetes.api.model.KubernetesList["items"]->java.util.ArrayList[0])
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
	at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:867)
	at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:78)
	at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:32)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:258)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
	at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3708)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2005)
	at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:2476)
	at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:80)
	at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:32)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2726)
	at io.fabric8.kubernetes.client.dsl.internal.WatchHTTPManager.readWatchEvent(WatchHTTPManager.java:312)
	at io.fabric8.kubernetes.client.dsl.internal.WatchHTTPManager.onMessage(WatchHTTPManager.java:247)
	at io.fabric8.kubernetes.client.dsl.internal.WatchHTTPManager$2.onResponse(WatchHTTPManager.java:176)
	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

The only way to make it work is to keep @ljnelson 's workaround.

[rohanKanojia]

@xguerin : ah, okay. Let me take this up myself

[mgorniew]

@xguerin I'm not sure which workaround you have in mind. KubernetesDeserializer.registerCustomKind? This is not workaround, this step is needed. Something like this should work:

        CustomResourceDefinitionNames names = new CustomResourceDefinitionNames();
        names.setSingular("export");
        names.setPlural("exports");
        names.setKind("Export");

        CustomResourceDefinitionSpec spec = new CustomResourceDefinitionSpec();
        spec.setNames(names);
        spec.setGroup("my.controller.com");
        spec.setVersion("v1alpha1");
        spec.setScope("Namespaced");

        CustomResourceDefinition customResourceDefinition = new CustomResourceDefinition();
        customResourceDefinition.setSpec(spec);

        KubernetesDeserializer.registerCustomKind("my.controller.com/v1alpha1", "Export", Export.class);

        new DefaultKubernetesClient().customResource(customResourceDefinition, Export.class, ExportList.class, DoneableExport.class)
            .inNamespace("test").watch(new ExportWatcher());

[xguerin]

@mgorniew

This is not workaround, this step is needed.

Ahhhhh, ok. I think that detail was not abundantly clear in my head. Thanks for clarifying!

[ljnelson]

KubernetesDeserializer is in a package with internal in the name. Seems odd I’m supposed to use it, right?

[rohanKanojia]

I think we should move it from there #1285

[stale]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!