Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic classes lead to serialisation issues #153

Closed
upMKuhn opened this issue Aug 17, 2023 · 7 comments
Closed

Generic classes lead to serialisation issues #153

upMKuhn opened this issue Aug 17, 2023 · 7 comments

Comments

@upMKuhn
Copy link

upMKuhn commented Aug 17, 2023

Hello,

I have a class called DatedValue of type String and Double. The schema for this class is generated wrongly. As you can see the first
definition says the value is of type string and then this reference is wrongly repeated. Is this a bug in avro4k or a limitation in avro schemas?

Thank you!

java.lang.ClassCastException: value 638.2585562648774 (a java.lang.Double) cannot be cast to expected type string at [v1, v1, v1, v1, v1, v1, v1, v1][v1].owners[0].quantity[0].value
at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:79)
at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:30)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:84)
at com.github.avrokotlin.avro4k.io.DefaultAvroOutputStream.write(DefaultAvroOutputStream.kt:32)
at com.github.avrokotlin.avro4k.io.AvroOutputStream$DefaultImpls.write(AvroOutputStream.kt:22)
at com.github.avrokotlin.avro4k.io.DefaultAvroOutputStream.write(DefaultAvroOutputStream.kt:15)

@Serializable
data class Owner(
    val brokerId: String,
    val type: Int,
    val name: List<DatedValue<String>>,
    val quantity: List<DatedValue<Double>>
) {
    companion object
}

@Serializable
data class DatedValue<T>(
    @Serializable(with = LocalDateSerializer::class)
    val asOfDate: LocalDate,
    val value: T
) {
    companion object
}

Schema

        {
          "name": "owners",
          "type":
            {
              "type": "array",
              "items":
                {
                  "type": "record",
                  "name": "Owner",
                  "namespace": "com.mk.trading.broker.models.contracts.fundamentals",
                  "fields":
                    [
                      { "name": "brokerId", "type": "string" },
                      { "name": "type", "type": "int" },
                      {
                        "name": "name",
                        "type":
                          {
                            "type": "array",
                            "items":
                              {
                                "type": "record",
                                "name": "DatedValue",
                                "fields":
                                  [
                                    { "name": "asOfDate", "type": "string" },
                                    { "name": "value", "type": "string" },
                                  ],
                              },
                          },
                      },
                      {
                        "name": "quantity",
                        "type": { "type": "array", "items": "DatedValue" },
                      },
                    ],
                },
            },
        },
@thake
Copy link
Member

thake commented Aug 18, 2023

Hey @upMKuhn, this seems to be a bug in avro4k. Thanks for bringing it up. Can you provide a branch with a Junit-Testcase? This helps a lot to track down the cause.

@upMKuhn
Copy link
Author

upMKuhn commented Aug 18, 2023

Done, I added a schema test :)

fyi: @thake

#154

@thake
Copy link
Member

thake commented Aug 19, 2023

Thanks, I will have a look into it!

@thake
Copy link
Member

thake commented Aug 22, 2023

Hey @upMKuhn, I've dug a little bit deeper and found an Avro-Issue that states that Avro, as of now, does not support parameterized types. Until this issue is resolved, I wouldn't know how to correctly generate a schema for a parameterized type.

As a workaround, you could do the following:

```kotlin
@Serializable
data class Owner(
  val brokerId: String,
  val type: Int,
  val name: List<StringDatedValue>,
  val quantity: List<DoubleDatedValue>
)
interface DatedValue<T> {  
  val asOfDate: LocalDate
  val value:  T
}
@Serializable
data class StringDatedValue(
  @Serializable(with = LocalDateSerializer::class)
  override val asOfDate: LocalDate,
  override val value: String
) : DatedValue<String>

@Serializable
data class DoubleDatedValue(
  @Serializable(with = LocalDateSerializer::class)
  override val asOfDate: LocalDate,
  override val value: Double
) : DatedValue<Double>

@upMKuhn
Copy link
Author

upMKuhn commented Aug 22, 2023

Nice thank you!

@Chuckame
Copy link
Contributor

Chuckame commented Aug 22, 2023

But in parallel, kotlinx serialization should handle nested generics, another feature to take into account in #148

@thake
Copy link
Member

thake commented Aug 24, 2023

Closing this issue for now. If parameterized types are supported in avro, we'll open a new issue.

@thake thake closed this as not planned Won't fix, can't repro, duplicate, stale Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants