Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nested Avro schema raises exception trying to produce event #757

Closed
nhuber-tc opened this issue Jul 12, 2021 · 2 comments · Fixed by #778
Closed

Nested Avro schema raises exception trying to produce event #757

nhuber-tc opened this issue Jul 12, 2021 · 2 comments · Fixed by #778
Labels
backend Need a backend update bug Something isn't working topic data Kafka Topic data

Comments

@nhuber-tc
Copy link

When I try and produce a record through the UI with a nested schema, which has a union null/long type I get a traceback about the data not matching the schema. If I use other types, like strings, or don't have the union with null then it is fine. If I send a value larger than a 32 bit integer then it works (2147483648). It does seem to just be a problem with this service since I can produce data conforming to the schema with low-value integers without problem.

Data sent

{
  "inside": {
    "something": 1
  }
}

Traceback shown

Internal Server Error: Not in union {"type":"record","name":"inside_record","doc":"inside","fields":[{"name":"something","type":["null","long"],"default":null}]}: {"something": 1} (field=inside)
org.apache.avro.UnresolvedUnionException: Not in union {"type":"record","name":"inside_record","doc":"inside","fields":[{"name":"something","type":["null","long"],"default":null}]}: {"something": 1} (field=inside)
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:223)
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:210)
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:131)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
	at org.akhq.modules.AvroSerializer.fromJsonToAvro(AvroSerializer.java:99)
	at org.akhq.modules.AvroSerializer.toAvro(AvroSerializer.java:70)
	at org.akhq.repositories.RecordRepository.produce(RecordRepository.java:530)
	at org.akhq.controllers.TopicController.produce(TopicController.java:141)
	at org.akhq.controllers.$TopicControllerDefinition$$exec4.invokeInternal(Unknown Source)
	at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:151)
	at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:482)
	at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303)
	at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:121)
	at io.micronaut.http.server.netty.RoutingInBoundHandler.emitRouteResponse(RoutingInBoundHandler.java:1533)
	at io.micronaut.http.server.netty.RoutingInBoundHandler.access$2100(RoutingInBoundHandler.java:151)
	at io.micronaut.http.server.netty.RoutingInBoundHandler$6$1.request(RoutingInBoundHandler.java:1500)
	at io.reactivex.internal.subscriptions.SubscriptionHelper.deferredSetOnce(SubscriptionHelper.java:202)
	at io.reactivex.internal.operators.single.SingleFlatMapPublisher$SingleFlatMapPublisherObserver.onSubscribe(SingleFlatMapPublisher.java:107)
	at io.micronaut.http.server.netty.RoutingInBoundHandler$6.doSubscribe(RoutingInBoundHandler.java:1488)
	at io.micronaut.http.server.netty.RoutingInBoundHandler$6.lambda$subscribe$0(RoutingInBoundHandler.java:1482)
	at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:79)
	at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:157)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
	Suppressed: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1 (field=something)
		at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:223)
		at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:210)
		at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:131)
		at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83)
		at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:221)
		... 25 more
		Suppressed: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1
			at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:877)
			at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:272)
			at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:143)
			at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83)
			at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:221)
			... 29 more

Schema:

{
  "type": "record",
  "name": "Outside",
  "doc": "outside",
  "fields": [
    {
      "name": "inside",
      "type": {
        "type": "record",
        "name": "inside_record",
        "doc": "inside",
        "fields": [
          {
            "name": "something",
            "type": [
              "null",
              "long"
            ],
            "default": null
          }
        ]
      }
    }
  ]
}
@tchiotludo
Copy link
Owner

What version of akhq are you using ?

@nhuber-tc
Copy link
Author

0.18.0, running from the tchiotludo/akhq:latest docker image and pulled yesterday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Need a backend update bug Something isn't working topic data Kafka Topic data
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants