JerseyEurekaHttpClient issue with 404 responses in Spring Boot 3.x

[kvmw]

Spring Boot 3.x includes a payload to error responses, by default.
exmple:

{
  "timestamp":"<time>",
  "status": 404,
  "error": "Not Found",
  "path": "/eureka/apps/<app-id>/<instance-id>"
}

On the other hand JerseyEurekaHttpClient::sendHeartBeat method tries to read and deserialise the response payload to InstanceInfo , if it is not empty.

Failure in deserialising the above error payload results in 400 (bad request) error and the client does not continue with the registration (which supposed to happened after receiving 404).

A similar issue in Spring Cloud Netflix has been fixed by updating RestTemplateTransportClientFactory and WebClientTransportClientFactory.

I believe a simple solution needs to be implemented for JerseyEurekaHttpClient.

Here is a sample stacktrace :

2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT 2023-09-18T14:12:23.518Z  INFO 7 --- [tbeatExecutor-0] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='https://SERVER/eureka/}, exception=HTTP 400 Bad Request stacktrace=jakarta.ws.rs.WebApplicationException: HTTP 400 Bad Request
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.provider.DiscoveryJerseyProvider.readFrom(DiscoveryJerseyProvider.java:110)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:233)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:212)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:132)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.spi.ContentEncoder.aroundReadFrom(ContentEncoder.java:102)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:132)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1072)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:919)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:853)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.client.ClientResponse.readEntity(ClientResponse.java:298)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.client.InboundJaxrsResponse$1.call(InboundJaxrsResponse.java:93)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:365)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.client.InboundJaxrsResponse.runInScopeIfPossible(InboundJaxrsResponse.java:244)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.glassfish.jersey.client.InboundJaxrsResponse.readEntity(InboundJaxrsResponse.java:90)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.jersey3.AbstractJersey3EurekaHttpClient.sendHeartBeat(AbstractJersey3EurekaHttpClient.java:151)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.MetricsCollectingEurekaHttpClient.execute(MetricsCollectingEurekaHttpClient.java:73)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.executeOnNewServer(RedirectingEurekaHttpClient.java:121)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.execute(RedirectingEurekaHttpClient.java:80)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:120)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.DiscoveryClient.renew(DiscoveryClient.java:837)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.DiscoveryClient$HeartbeatThread.run(DiscoveryClient.java:1401)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at java.base/java.lang.Thread.run(Unknown Source)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Root name ('timestamp') does not match expected ('instance') for type `com.netflix.appinfo.InstanceInfo`
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 2] (through reference chain: com.netflix.appinfo.InstanceInfo["timestamp"])
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.DeserializationContext.reportPropertyInputMismatch(DeserializationContext.java:1787)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.DeserializationContext.reportPropertyInputMismatch(DeserializationContext.java:1803)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext._unwrapAndDeserialize(DefaultDeserializationContext.java:348)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:320)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2105)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1481)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.converters.EurekaJacksonCodec.readValue(EurekaJacksonCodec.java:213)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at org.springframework.cloud.netflix.eureka.server.CloudJacksonJson.decode(CloudJacksonJson.java:84)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT at com.netflix.discovery.provider.DiscoveryJerseyProvider.readFrom(DiscoveryJerseyProvider.java:103)
   2023-09-18T16:12:23.51+0200 [APP/PROC/WEB/0] OUT ... 37 more

[ouaibsky]

Hello

I confirm this is a big issue when using spring-cloud-eureka-server, breaking the replication between 2 server instances (after one instance disappear and try to come back)
more information there: spring-cloud/spring-cloud-netflix#4220
It contains an example on how to reproduce.

It's a simple fix I can do but I'm not able to find the source code for eureka-client-jersey3 ?
Any clue please ?

pom.xml from eureak-client-jersey3 is referencing this repository.
I'm unable to find any repository or source code in github for AbstractJersey3EurekaHttpClient.java

Thx
Christophe

[ouaibsky]

Hey
Does someone can help me by explaining me where is eureka-client-jersey3 sources ?
Then I'll be able to propose PR.

[spencergibb]

https://github.com/Netflix/eureka/tree/2.x

[ouaibsky]

PR is available here: #1540

Tested successfully with example provided at: spring-cloud/spring-cloud-netflix#4220 (comment)