diff --git a/Migrating-from-v1.md b/Migrating-from-v1.md new file mode 100644 index 00000000..4ab3316d --- /dev/null +++ b/Migrating-from-v1.md @@ -0,0 +1,117 @@ +Here is the guide of how to migrate from Avro4k v1 to v2 using examples. + +> [!INFO] +> If you are missing a migration need, please [file an issue](https://github.com/avro-kotlin/avro4k/issues/new/choose) or [make a PR](https://github.com/avro-kotlin/avro4k/compare). + +## Pure avro serialization + +```kotlin +// Previously +val bytes = Avro.default.encodeToByteArray(TheDataClass.serializer(), TheDataClass(...)) +Avro.default.decodeFromByteArray(TheDataClass.serializer(), bytes) + +// Now +val bytes = Avro.encodeToByteArray(TheDataClass(...)) +Avro.decodeFromByteArray(bytes) +``` + +## Set a field default value to null + +```kotlin +// Previously +data class TheDataClass( + @AvroDefault(Avro.NULL) + val field: String? +) + +// Now +// ... Nothing, as it is the default behavior! +data class TheDataClass( + val field: String? +) +``` + +## generic data serialization +Convert a kotlin data class to a `GenericRecord` to then be handled by a `GenericDatumWriter` in avro. + +```kotlin +// Previously +val genericRecord: GenericRecord = Avro.default.toRecord(TheDataClass.serializer(), TheDataClass(...)) +Avro.default.fromRecord(TheDataClass.serializer(), genericRecord) + +// Now +val genericData: Any? = Avro.encodeToGenericData(TheDataClass(...)) +Avro.decodeFromGenericData(genericData) +``` + + +## Configure the `Avro` instance + +```kotlin +// Previously +val avro = Avro( + AvroConfiguration( + namingStrategy = FieldNamingStrategy.SnackCase, + implicitNulls = true, + ), + SerializersModule { + contextual(CustomSerializer()) + } +) + +// Now +val avro = Avro { + namingStrategy = FieldNamingStrategy.SnackCase + implicitNulls = true + serializersModule = SerializersModule { + contextual(CustomSerializer()) + } +} +``` + +## Changing the name of a record + +```kotlin +// Previously +@AvroName("TheName") +@AvroNamespace("a.custom.namespace") +data class TheDataClass(...) + +// Now +@SerialName("a.custom.namespace.TheName") +data class TheDataClass(...) +``` + +## Writing an avro object container file with a custom field naming strategy + +```kotlin +// Previously +Files.newOutputStream(Path("/your/file.avro")).use { outputStream -> + Avro(AvroConfiguration(namingStrategy = SnakeCaseNamingStrategy)) + .openOutputStream(TheDataClass.serializer()) { encodeFormat = AvroEncodeFormat.Data(CodecFactory.snappyCodec()) } + .to(outputStream) + .write(TheDataClass(...)) + .write(TheDataClass(...)) + .write(TheDataClass(...)) + .close() +} + + +// Now +val dataSequence = sequenceOf( + TheDataClass(...), + TheDataClass(...), + TheDataClass(...), +) +val avro = Avro { fieldNamingStrategy = FieldNamingStrategy.SnakeCase } +Files.newOutputStream(Path("/your/file.avro")).use { outputStream -> + AvroObjectContainerFile(avro) + .encodeToStream(dataSequence, outputStream) { + codec(CodecFactory.snappyCodec()) + // you can also add your metadata ! + metadata("myProp", 1234L) + metadata("a string metadata", "hello") + } +} +``` + diff --git a/README.md b/README.md index db51acbd..a807bd6e 100644 --- a/README.md +++ b/README.md @@ -136,7 +136,7 @@ fun main() { - **Avro4k** is highly based on apache avro library, that implies all the schema validation is done by it - All members annotated with `@ExperimentalSerializationApi` are **subject to changes** in future releases without any notice as they are experimental, so please - check the release notes to check the needed migration + check the release notes to check the needed migration. At least, given a version `A.B.C`, only the minor `B` number will be incremented, not the major `A`. - **Avro4k** also supports encoding and decoding generic data, mainly because of confluent schema registry compatibility as their serializers only handle generic data. When avro4k will support their schema registry, the generic encoding will be removed to keep this library as simple as possible. @@ -737,30 +737,6 @@ package my.package data class MyData(val myField: String) ``` -## Changing the namespace of all nested named type(s) - -Sometimes, using classes from other packages or libraries, you may want to change the namespace of a nested named type. This is done annotating the field -with `@AvroNamespaceOverride`. - -```kotlin -import kotlinx.serialization.Serializable -import com.github.avrokotlin.avro4k.AvroNamespaceOverride - -@Serializable -data class MyData( - @AvroNamespaceOverride("new.namespace") val myField: NestedRecord -) - -// ... -package external.package.name - -@Serializable -data class NestedRecord(val field: String) -``` - -> [!NOTE] -> This impacts the schema generation, the serialization and the deserialization. - ## Change type name (FIXED only) > [!WARNING] @@ -836,6 +812,9 @@ So to mark a field as optional and facilitate avro contract evolution regarding - Kotlin 1.7.20 up to 1.8.10 cannot properly compile @SerialInfo-Annotations on enums (see https://github.com/Kotlin/kotlinx.serialization/issues/2121). This is fixed with kotlin 1.8.20. So if you are planning to use any of avro4k's annotations on enum types, please make sure that you are using kotlin >= 1.8.20. +# Migrating from v1 to v2 +Heads up to the [migration guide](Migrating-from-v1.md) to update your code from avro4k v1 to v2. + # Contributions Contributions to avro4k are always welcome. Good ways to contribute include: diff --git a/api/avro4k-core.api b/api/avro4k-core.api index 0f6885ff..8cef4b4f 100644 --- a/api/avro4k-core.api +++ b/api/avro4k-core.api @@ -1,14 +1,20 @@ +public final class com/github/avrokotlin/avro4k/AnnotationsKt { + public static final fun asAvroLogicalType (Lkotlinx/serialization/descriptors/SerialDescriptor;)Lkotlinx/serialization/descriptors/SerialDescriptor; +} + public abstract interface class com/github/avrokotlin/avro4k/AnyValueDecoder { public abstract fun decodeAny (Lcom/github/avrokotlin/avro4k/AvroDecoder;)Ljava/lang/Object; } -public abstract class com/github/avrokotlin/avro4k/Avro { +public abstract class com/github/avrokotlin/avro4k/Avro : kotlinx/serialization/BinaryFormat { public static final field Default Lcom/github/avrokotlin/avro4k/Avro$Default; public synthetic fun (Lcom/github/avrokotlin/avro4k/AvroConfiguration;Lkotlinx/serialization/modules/SerializersModule;Lkotlin/jvm/internal/DefaultConstructorMarker;)V + public fun decodeFromByteArray (Lkotlinx/serialization/DeserializationStrategy;[B)Ljava/lang/Object; public final fun decodeFromByteArray (Lorg/apache/avro/Schema;Lkotlinx/serialization/DeserializationStrategy;[B)Ljava/lang/Object; + public fun encodeToByteArray (Lkotlinx/serialization/SerializationStrategy;Ljava/lang/Object;)[B public final fun encodeToByteArray (Lorg/apache/avro/Schema;Lkotlinx/serialization/SerializationStrategy;Ljava/lang/Object;)[B public final fun getConfiguration ()Lcom/github/avrokotlin/avro4k/AvroConfiguration; - public final fun getSerializersModule ()Lkotlinx/serialization/modules/SerializersModule; + public fun getSerializersModule ()Lkotlinx/serialization/modules/SerializersModule; public final fun schema (Lkotlinx/serialization/descriptors/SerialDescriptor;)Lorg/apache/avro/Schema; } @@ -24,10 +30,6 @@ public synthetic class com/github/avrokotlin/avro4k/AvroAlias$Impl : com/github/ public final synthetic fun value ()[Ljava/lang/String; } -public final class com/github/avrokotlin/avro4k/AvroAnnotationsKt { - public static final fun asAvroLogicalType (Lkotlinx/serialization/descriptors/SerialDescriptor;)Lkotlinx/serialization/descriptors/SerialDescriptor; -} - public final class com/github/avrokotlin/avro4k/AvroBuilder { public final fun build ()Lcom/github/avrokotlin/avro4k/AvroConfiguration; public final fun getFieldNamingStrategy ()Lcom/github/avrokotlin/avro4k/FieldNamingStrategy; @@ -164,15 +166,6 @@ public final class com/github/avrokotlin/avro4k/AvroKt { public static final fun schema (Lcom/github/avrokotlin/avro4k/Avro;Lkotlinx/serialization/KSerializer;)Lorg/apache/avro/Schema; } -public abstract interface annotation class com/github/avrokotlin/avro4k/AvroNamespaceOverride : java/lang/annotation/Annotation { - public abstract fun value ()Ljava/lang/String; -} - -public synthetic class com/github/avrokotlin/avro4k/AvroNamespaceOverride$Impl : com/github/avrokotlin/avro4k/AvroNamespaceOverride { - public fun (Ljava/lang/String;)V - public final synthetic fun value ()Ljava/lang/String; -} - public final class com/github/avrokotlin/avro4k/AvroObjectContainerFile { public fun ()V public fun (Lcom/github/avrokotlin/avro4k/Avro;)V @@ -224,16 +217,18 @@ public synthetic class com/github/avrokotlin/avro4k/AvroProp$Impl : com/github/a public final synthetic fun value ()Ljava/lang/String; } -public final class com/github/avrokotlin/avro4k/AvroSingleObject { +public final class com/github/avrokotlin/avro4k/AvroSingleObject : kotlinx/serialization/BinaryFormat { public fun (Lkotlin/jvm/functions/Function1;Lcom/github/avrokotlin/avro4k/Avro;)V public synthetic fun (Lkotlin/jvm/functions/Function1;Lcom/github/avrokotlin/avro4k/Avro;ILkotlin/jvm/internal/DefaultConstructorMarker;)V + public fun decodeFromByteArray (Lkotlinx/serialization/DeserializationStrategy;[B)Ljava/lang/Object; public final fun decodeFromStream (Lkotlinx/serialization/DeserializationStrategy;Ljava/io/InputStream;)Ljava/lang/Object; + public fun encodeToByteArray (Lkotlinx/serialization/SerializationStrategy;Ljava/lang/Object;)[B public final fun encodeToStream (Lorg/apache/avro/Schema;Lkotlinx/serialization/SerializationStrategy;Ljava/lang/Object;Ljava/io/OutputStream;)V public final fun getAvro ()Lcom/github/avrokotlin/avro4k/Avro; + public fun getSerializersModule ()Lkotlinx/serialization/modules/SerializersModule; } public final class com/github/avrokotlin/avro4k/AvroSingleObjectKt { - public static final fun decodeFromByteArray (Lcom/github/avrokotlin/avro4k/AvroSingleObject;Lkotlinx/serialization/DeserializationStrategy;[B)Ljava/lang/Object; public static final fun encodeToByteArray (Lcom/github/avrokotlin/avro4k/AvroSingleObject;Lorg/apache/avro/Schema;Lkotlinx/serialization/SerializationStrategy;Ljava/lang/Object;)[B } diff --git a/benchmark/src/main/kotlin/com/github/avrokotlin/benchmark/Avro4kClientsBenchmark.kt b/benchmark/src/main/kotlin/com/github/avrokotlin/benchmark/Avro4kClientsBenchmark.kt index 80e1cc71..d92e54a1 100644 --- a/benchmark/src/main/kotlin/com/github/avrokotlin/benchmark/Avro4kClientsBenchmark.kt +++ b/benchmark/src/main/kotlin/com/github/avrokotlin/benchmark/Avro4kClientsBenchmark.kt @@ -1,9 +1,9 @@ package com.github.avrokotlin.benchmark import com.github.avrokotlin.avro4k.Avro -import com.github.avrokotlin.avro4k.decodeFromByteArray -import com.github.avrokotlin.avro4k.encodeToByteArray import kotlinx.benchmark.Benchmark +import kotlinx.serialization.decodeFromByteArray +import kotlinx.serialization.encodeToByteArray internal object Avro4kClientsStaticReadBenchmark { @JvmStatic diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroAnnotations.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/Annotations.kt similarity index 88% rename from src/main/kotlin/com/github/avrokotlin/avro4k/AvroAnnotations.kt rename to src/main/kotlin/com/github/avrokotlin/avro4k/Annotations.kt index 84334125..7dc02b36 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroAnnotations.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/Annotations.kt @@ -11,18 +11,6 @@ import kotlinx.serialization.descriptors.SerialDescriptor import org.apache.avro.LogicalType import org.intellij.lang.annotations.Language -/** - * When annotated on a property, deeply overrides the namespace for all the nested named types (records, enums and fixed). - * - * Works with standard classes and inline classes. - */ -@SerialInfo -@ExperimentalSerializationApi -@Target(AnnotationTarget.PROPERTY) -public annotation class AvroNamespaceOverride( - val value: String, -) - /** * Adds a property to the Avro schema or field. Its value could be any valid JSON or just a string. * @@ -83,6 +71,11 @@ public annotation class AvroFixed(val size: Int) /** * Sets the default avro value for a record's field. * + * - Records and maps have to be represented as a json object + * - Arrays have to be represented as a json array + * - Nulls have to be represented as a json `null`. To set the string `"null"`, don't forget to quote the string, example: `""""null""""` or `"\"null\""`. + * - Any non json content will be treated as a string + * * Ignored in inline classes. */ @SerialInfo @@ -112,7 +105,7 @@ public annotation class AvroEnumDefault * } * ``` * - * For more complex needs, please file an issue [here](https://github.com/avro-kotlin/avro4k/issues). + * For more complex needs, please file an feature request [here](https://github.com/avro-kotlin/avro4k/issues). */ @ExperimentalSerializationApi public fun SerialDescriptor.asAvroLogicalType(): SerialDescriptor { diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/Avro.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/Avro.kt index 85f5591a..29ee63bd 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/Avro.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/Avro.kt @@ -12,6 +12,7 @@ import com.github.avrokotlin.avro4k.serializer.LocalDateTimeSerializer import com.github.avrokotlin.avro4k.serializer.LocalTimeSerializer import com.github.avrokotlin.avro4k.serializer.URLSerializer import com.github.avrokotlin.avro4k.serializer.UUIDSerializer +import kotlinx.serialization.BinaryFormat import kotlinx.serialization.DeserializationStrategy import kotlinx.serialization.ExperimentalSerializationApi import kotlinx.serialization.KSerializer @@ -33,15 +34,15 @@ import java.io.ByteArrayInputStream */ public sealed class Avro( public val configuration: AvroConfiguration, - public val serializersModule: SerializersModule, -) { + public override val serializersModule: SerializersModule, +) : BinaryFormat { // We use the identity hash map because we could have multiple descriptors with the same name, especially - // when having 2 different version of the schema for the same name. kotlinx-serialization is instanciating the descriptors + // when having 2 different version of the schema for the same name. kotlinx-serialization is instantiating the descriptors // only once, so we are safe in the main use cases. Combined with weak references to avoid memory leaks. private val schemaCache: MutableMap = WeakIdentityHashMap() internal val recordResolver = RecordResolver(this) - internal val polymorphicResolver = PolymorphicResolver(this.serializersModule) + internal val polymorphicResolver = PolymorphicResolver(serializersModule) internal val enumResolver = EnumResolver() public companion object Default : Avro( @@ -88,6 +89,20 @@ public sealed class Avro( } return result } + + override fun decodeFromByteArray( + deserializer: DeserializationStrategy, + bytes: ByteArray, + ): T { + return decodeFromByteArray(schema(deserializer.descriptor), deserializer, bytes) + } + + override fun encodeToByteArray( + serializer: SerializationStrategy, + value: T, + ): ByteArray { + return encodeToByteArray(schema(serializer.descriptor), serializer, value) + } } public fun Avro( @@ -121,8 +136,6 @@ public class AvroBuilder internal constructor(avro: Avro) { private class AvroImpl(configuration: AvroConfiguration, serializersModule: SerializersModule) : Avro(configuration, serializersModule) -// schema gen extensions - public inline fun Avro.schema(): Schema { val serializer = serializersModule.serializer() return schema(serializer.descriptor) @@ -132,13 +145,6 @@ public fun Avro.schema(serializer: KSerializer): Schema { return schema(serializer.descriptor) } -// encoding extensions - -public inline fun Avro.encodeToByteArray(value: T): ByteArray { - val serializer = serializersModule.serializer() - return encodeToByteArray(schema(serializer), serializer, value) -} - public inline fun Avro.encodeToByteArray( writerSchema: Schema, value: T, @@ -147,13 +153,6 @@ public inline fun Avro.encodeToByteArray( return encodeToByteArray(writerSchema, serializer, value) } -// decoding extensions - -public inline fun Avro.decodeFromByteArray(bytes: ByteArray): T { - val serializer = serializersModule.serializer() - return decodeFromByteArray(schema(serializer.descriptor), serializer, bytes) -} - public inline fun Avro.decodeFromByteArray( writerSchema: Schema, bytes: ByteArray, diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroDecoder.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/AvroDecoder.kt index f03a4da5..482ac111 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroDecoder.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/AvroDecoder.kt @@ -45,9 +45,19 @@ public interface AvroDecoder : Decoder { @ExperimentalSerializationApi public val currentWriterSchema: Schema + /** + * Decode a [Schema.Type.BYTES] value. + * + * A bytes value is a sequence of bytes prefixed with an int corresponding to its length. + */ @ExperimentalSerializationApi public fun decodeBytes(): ByteArray + /** + * Decode a [Schema.Type.FIXED] value. + * + * A fixed value is a fixed-size sequence of bytes, where the length is not materialized in the binary output as it is known by the [currentWriterSchema]. + */ @ExperimentalSerializationApi public fun decodeFixed(): GenericFixed diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroEncoder.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/AvroEncoder.kt index 08e3a299..cf4eaddc 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroEncoder.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/AvroEncoder.kt @@ -33,15 +33,27 @@ public interface AvroEncoder : Encoder { @ExperimentalSerializationApi public val currentWriterSchema: Schema + /** + * Encodes a [Schema.Type.BYTES] value from a [ByteBuffer]. + */ @ExperimentalSerializationApi public fun encodeBytes(value: ByteBuffer) + /** + * Encodes a [Schema.Type.BYTES] value from a [ByteArray]. + */ @ExperimentalSerializationApi public fun encodeBytes(value: ByteArray) + /** + * Encodes a [Schema.Type.FIXED] value from a [ByteArray]. Its size must match the size of the fixed schema in [currentWriterSchema]. + */ @ExperimentalSerializationApi public fun encodeFixed(value: ByteArray) + /** + * Encodes a [Schema.Type.FIXED] value from a [GenericFixed]. Its size must match the size of the fixed schema in [currentWriterSchema]. + */ @ExperimentalSerializationApi public fun encodeFixed(value: GenericFixed) } diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroSingleObject.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/AvroSingleObject.kt index d576c602..3ca113fe 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/AvroSingleObject.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/AvroSingleObject.kt @@ -1,9 +1,11 @@ package com.github.avrokotlin.avro4k +import kotlinx.serialization.BinaryFormat import kotlinx.serialization.DeserializationStrategy import kotlinx.serialization.ExperimentalSerializationApi import kotlinx.serialization.SerializationException import kotlinx.serialization.SerializationStrategy +import kotlinx.serialization.modules.SerializersModule import kotlinx.serialization.serializer import org.apache.avro.Schema import org.apache.avro.SchemaNormalization @@ -28,7 +30,10 @@ public class AvroSingleObject( private val schemaRegistry: (fingerprint: Long) -> Schema?, @PublishedApi internal val avro: Avro = Avro, -) { +) : BinaryFormat { + override val serializersModule: SerializersModule + get() = avro.serializersModule + private fun Schema.crc64avro(): ByteArray = SchemaNormalization.parsingFingerprint("CRC-64-AVRO", this) public fun encodeToStream( @@ -55,6 +60,22 @@ public class AvroSingleObject( return avro.decodeFromStream(writerSchema, deserializer, inputStream) } + + public override fun decodeFromByteArray( + deserializer: DeserializationStrategy, + bytes: ByteArray, + ): T { + return bytes.inputStream().use { + decodeFromStream(deserializer, it) + } + } + + override fun encodeToByteArray( + serializer: SerializationStrategy, + value: T, + ): ByteArray { + return encodeToByteArray(avro.schema(serializer.descriptor), serializer, value) + } } private const val MAGIC_BYTE: Int = 0xC3 @@ -79,12 +100,4 @@ public inline fun AvroSingleObject.encodeToByteArray(value: T): Byte return encodeToByteArray(avro.schema(serializer), serializer, value) } -public fun AvroSingleObject.decodeFromByteArray( - deserializer: DeserializationStrategy, - bytes: ByteArray, -): T = - bytes.inputStream().use { - decodeFromStream(deserializer, it) - } - public inline fun AvroSingleObject.decodeFromByteArray(bytes: ByteArray): T = decodeFromByteArray(avro.serializersModule.serializer(), bytes) \ No newline at end of file diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/ClassVisitor.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/ClassVisitor.kt index dfd3661e..f36c6e99 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/ClassVisitor.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/ClassVisitor.kt @@ -4,7 +4,6 @@ import com.github.avrokotlin.avro4k.AvroDefault import com.github.avrokotlin.avro4k.internal.isStartingAsJson import com.github.avrokotlin.avro4k.internal.jsonNode import com.github.avrokotlin.avro4k.internal.nonNullSerialName -import com.github.avrokotlin.avro4k.internal.overrideNamespace import kotlinx.serialization.SerializationException import kotlinx.serialization.builtins.serializer import kotlinx.serialization.descriptors.SerialDescriptor @@ -99,25 +98,20 @@ internal class ClassVisitor( elementDescriptor: SerialDescriptor, elementSchema: Schema, ): Schema.Field { - var finalSchema: Schema = annotations.namespaceOverride?.value?.let { elementSchema.overrideNamespace(it) } ?: elementSchema + val fieldDefault = getFieldDefault(annotations.default, elementSchema, elementDescriptor) - val fieldDefault = getFieldDefault(annotations.default, finalSchema, elementDescriptor) - - if (fieldDefault != null) { - reorderUnionIfNeeded(fieldDefault, finalSchema)?.let { - finalSchema = it + val finalSchema = + if (fieldDefault != null) { + reorderUnionIfNeeded(fieldDefault, elementSchema) + } else { + elementSchema } - } val field = Schema.Field( - // name = fieldName, - // schema = finalSchema, - // doc = annotations.doc?.value, - // defaultValue = fieldDefault ) annotations.aliases.flatMap { it.value.asSequence() }.forEach { field.addAlias(it) } @@ -130,9 +124,9 @@ internal class ClassVisitor( */ private fun reorderUnionIfNeeded( fieldDefault: Any, - finalSchema: Schema, - ): Schema? { - if (finalSchema.isUnion && finalSchema.isNullable) { + schema: Schema, + ): Schema { + if (schema.isUnion && schema.isNullable) { var nullNotFirst = false if (fieldDefault is Collection<*>) { nullNotFirst = fieldDefault.any { it != JsonProperties.NULL_VALUE } @@ -140,13 +134,13 @@ internal class ClassVisitor( nullNotFirst = true } if (nullNotFirst) { - val nullIndex = finalSchema.types.indexOfFirst { it.type == Schema.Type.NULL } - val nonNullTypes = finalSchema.types.toMutableList() + val nullIndex = schema.types.indexOfFirst { it.type == Schema.Type.NULL } + val nonNullTypes = schema.types.toMutableList() val nullType = nonNullTypes.removeAt(nullIndex) return Schema.createUnion(nonNullTypes + nullType) } } - return null + return schema } private fun getFieldDefault( diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/InlineClassVisitor.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/InlineClassVisitor.kt index e82cf98f..d27c1489 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/InlineClassVisitor.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/InlineClassVisitor.kt @@ -2,7 +2,6 @@ package com.github.avrokotlin.avro4k.internal.schema import com.github.avrokotlin.avro4k.internal.copy import com.github.avrokotlin.avro4k.internal.jsonNode -import com.github.avrokotlin.avro4k.internal.overrideNamespace import kotlinx.serialization.descriptors.SerialDescriptor import org.apache.avro.Schema @@ -24,14 +23,13 @@ internal class InlineClassVisitor( return ValueVisitor(context.copy(inlinedAnnotations = inlinedAnnotations)) { generatedSchema -> val annotations = InlineClassFieldAnnotations(inlineClassDescriptor) - var schema = generatedSchema - if (annotations.namespaceOverride != null) { - schema = schema.overrideNamespace(annotations.namespaceOverride.value) - } val props = annotations.props.toList() - if (props.isNotEmpty()) { - schema = schema.copy(additionalProps = props.associate { it.key to it.jsonNode }) - } + val schema = + if (props.isNotEmpty()) { + generatedSchema.copy(additionalProps = props.associate { it.key to it.jsonNode }) + } else { + generatedSchema + } onSchemaBuilt(schema) } diff --git a/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/VisitorContext.kt b/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/VisitorContext.kt index e7435b95..caaf77c4 100644 --- a/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/VisitorContext.kt +++ b/src/main/kotlin/com/github/avrokotlin/avro4k/internal/schema/VisitorContext.kt @@ -5,7 +5,6 @@ import com.github.avrokotlin.avro4k.AvroAlias import com.github.avrokotlin.avro4k.AvroDefault import com.github.avrokotlin.avro4k.AvroDoc import com.github.avrokotlin.avro4k.AvroFixed -import com.github.avrokotlin.avro4k.AvroNamespaceOverride import com.github.avrokotlin.avro4k.AvroProp import com.github.avrokotlin.avro4k.internal.AnnotatedLocation import com.github.avrokotlin.avro4k.internal.findAnnotation @@ -29,11 +28,9 @@ internal fun VisitorContext.resetNesting() = copy(inlinedAnnotations = null) * Contains all the annotations for a field of a class (kind == CLASS && isInline == true). */ internal data class InlineClassFieldAnnotations( - val namespaceOverride: AvroNamespaceOverride?, val props: Sequence, ) { constructor(inlineClassDescriptor: SerialDescriptor) : this( - inlineClassDescriptor.findElementAnnotation(0), sequence { yieldAll(inlineClassDescriptor.findAnnotations()) yieldAll(inlineClassDescriptor.findElementAnnotations(0)) @@ -53,14 +50,12 @@ internal data class FieldAnnotations( val aliases: Sequence, val doc: AvroDoc?, val default: AvroDefault?, - val namespaceOverride: AvroNamespaceOverride?, ) { constructor(descriptor: SerialDescriptor, elementIndex: Int) : this( descriptor.findElementAnnotations(elementIndex).asSequence(), descriptor.findElementAnnotations(elementIndex).asSequence(), descriptor.findElementAnnotation(elementIndex), - descriptor.findElementAnnotation(elementIndex), - descriptor.findElementAnnotation(elementIndex) + descriptor.findElementAnnotation(elementIndex) ) { require(descriptor.kind == StructureKind.CLASS) { "${FieldAnnotations::class.qualifiedName} is only for classes, but trying at element index $elementIndex with non class descriptor $descriptor" diff --git a/src/test/kotlin/com/github/avrokotlin/avro4k/AvroSingleObjectTest.kt b/src/test/kotlin/com/github/avrokotlin/avro4k/AvroSingleObjectTest.kt index 73ef80bc..2fdd1c16 100644 --- a/src/test/kotlin/com/github/avrokotlin/avro4k/AvroSingleObjectTest.kt +++ b/src/test/kotlin/com/github/avrokotlin/avro4k/AvroSingleObjectTest.kt @@ -4,6 +4,7 @@ import io.kotest.core.spec.style.StringSpec import io.kotest.matchers.shouldBe import kotlinx.serialization.Contextual import kotlinx.serialization.Serializable +import kotlinx.serialization.encodeToByteArray import org.apache.avro.SchemaNormalization import java.time.Instant diff --git a/src/test/kotlin/com/github/avrokotlin/avro4k/encoding/AvroFixedEncodingTest.kt b/src/test/kotlin/com/github/avrokotlin/avro4k/encoding/AvroFixedEncodingTest.kt index f5d6c0d8..8cd9f093 100644 --- a/src/test/kotlin/com/github/avrokotlin/avro4k/encoding/AvroFixedEncodingTest.kt +++ b/src/test/kotlin/com/github/avrokotlin/avro4k/encoding/AvroFixedEncodingTest.kt @@ -3,7 +3,6 @@ package com.github.avrokotlin.avro4k.encoding import com.github.avrokotlin.avro4k.Avro import com.github.avrokotlin.avro4k.AvroAssertions import com.github.avrokotlin.avro4k.AvroFixed -import com.github.avrokotlin.avro4k.encodeToByteArray import com.github.avrokotlin.avro4k.record import com.github.avrokotlin.avro4k.schema import io.kotest.assertions.throwables.shouldThrow @@ -11,6 +10,7 @@ import io.kotest.core.spec.style.StringSpec import kotlinx.serialization.SerialName import kotlinx.serialization.Serializable import kotlinx.serialization.SerializationException +import kotlinx.serialization.encodeToByteArray import org.apache.avro.generic.GenericData import kotlin.io.path.Path diff --git a/src/test/kotlin/com/github/avrokotlin/avro4k/schema/AvroNamespaceOverrideSchemaTest.kt b/src/test/kotlin/com/github/avrokotlin/avro4k/schema/AvroNamespaceOverrideSchemaTest.kt deleted file mode 100644 index 88ec995c..00000000 --- a/src/test/kotlin/com/github/avrokotlin/avro4k/schema/AvroNamespaceOverrideSchemaTest.kt +++ /dev/null @@ -1,182 +0,0 @@ -package com.github.avrokotlin.avro4k.schema - -import com.github.avrokotlin.avro4k.Avro -import com.github.avrokotlin.avro4k.AvroAssertions -import com.github.avrokotlin.avro4k.AvroEnumDefault -import com.github.avrokotlin.avro4k.AvroFixed -import com.github.avrokotlin.avro4k.AvroNamespaceOverride -import com.github.avrokotlin.avro4k.schema -import io.kotest.core.spec.style.FunSpec -import io.kotest.matchers.shouldBe -import kotlinx.serialization.SerialName -import kotlinx.serialization.Serializable -import org.apache.avro.Schema -import org.apache.avro.SchemaBuilder -import kotlin.io.path.Path - -internal class AvroNamespaceOverrideSchemaTest : FunSpec({ - - test("support namespace annotations on records") { - val schema = Avro.schema() - schema.namespace shouldBe "com.yuval" - } - - test("support namespace annotations in nested records") { - AvroAssertions.assertThat() - .generatesSchema(Path("/namespace.json")) - } - - test("support namespace annotations on field") { - AvroAssertions.assertThat() - .generatesSchema(Path("/namespace.json")) - } - - test("favour namespace annotations on field over record") { - AvroAssertions.assertThat() - .generatesSchema(Path("/namespace.json")) - } - - test("favour namespace annotations on the topest value class field over record") { - AvroAssertions.assertThat() - .generatesSchema(Path("/namespace_nested.json")) - } - - test("empty namespace") { - AvroAssertions.assertThat() - .generatesSchema(Path("/namespace_empty.json")) - } - - test("support @AvroNamespaceOverride on map") { - val expected = - SchemaBuilder.map().values() - .record("AnnotatedNamespace").namespace("override").fields() - .name("s").type().stringType().noDefault() - .endRecord() - - AvroAssertions.assertThat() - .generatesSchema(expected) - } - - test("support @AvroNamespaceOverride on array") { - val expected = - SchemaBuilder.array().items() - .record("AnnotatedNamespace").namespace("override").fields() - .name("s").type().stringType().noDefault() - .endRecord() - - AvroAssertions.assertThat() - .generatesSchema(expected) - } - - test("support @AvroNamespaceOverride on union") { - val expected = - Schema.createUnion( - SchemaBuilder.record("One").namespace("override").fields() - .name("s").type().stringType().noDefault() - .endRecord(), - SchemaBuilder.record("Two").namespace("override").fields().endRecord() - ) - - AvroAssertions.assertThat() - .generatesSchema(expected) - } -}) { - @JvmInline - @Serializable - private value class MapNsOverride( - @AvroNamespaceOverride("override") val value: Map, - ) - - @JvmInline - @Serializable - private value class UnionNsOverride( - @AvroNamespaceOverride("override") val value: Root, - ) - - @Serializable - private sealed interface Root { - @Serializable - data class One(val s: String) : Root - - @Serializable - object Two : Root - } - - @JvmInline - @Serializable - private value class ArrayNsOverride( - @AvroNamespaceOverride("override") val value: List, - ) - - @SerialName("com.yuval.AnnotatedNamespace") - @Serializable - private data class AnnotatedNamespace(val s: String) - - @SerialName("com.yuval.internal.InternalAnnotated") - @Serializable - private data class InternalAnnotated(val i: Int) - - @SerialName("com.yuval.AnnotatedNamespace") - @Serializable - private data class NestedAnnotatedNamespace(val s: String, val internal: InternalAnnotated) - - @Serializable - @SerialName("InternalAnnotated") - private data class Internal(val i: Int) - - @Serializable - @SerialName("com.yuval.AnnotatedNamespace") - private data class InternalAnnotatedNamespace( - val s: String, - @AvroNamespaceOverride("com.yuval.internal") val internal: Internal, - ) - - @Serializable - @SerialName("shouldbeignored.InternalAnnotated") - private data class InternalIgnoreAnnotated(val i: Int) - - @Serializable - @SerialName("com.yuval.AnnotatedNamespace") - private data class FieldAnnotatedNamespace( - val s: String, - @AvroNamespaceOverride("com.yuval.internal") val internal: InternalIgnoreAnnotated, - ) - - @SerialName("Foo") - @Serializable - private data class Foo(val s: String) - - @SerialName("toto.NestedValueClasses") - @Serializable - private data class NestedValueClasses(val nested: Nested1) - - @JvmInline - @SerialName("shouldbeignored.Nested1") - @Serializable - private value class Nested1( - @AvroNamespaceOverride("primaryspace") val s: Nested2, - ) - - @JvmInline - @SerialName("shouldbeignored.Nested2") - @Serializable - private value class Nested2( - @AvroNamespaceOverride("shouldalsobeignored") val s: Nested3, - ) - - @SerialName("shouldbeignored.Nested3") - @Serializable - private data class Nested3( - val s: String, - @AvroNamespaceOverride("ignored") val enum: NestedEnum, - @AvroFixed(42) val fixed: String, - ) - - @Serializable - enum class NestedEnum { - A, - - @AvroEnumDefault - B, - } -} \ No newline at end of file