TypeMirror.asTypeName() returns java.lang.String when receiver type is kotlin.String

[hotchemi]

Overview

Thx for amazing library!

I encountered a problem that TypeMirror.asTypeName() returns java.lang.String when receiver type is kotlin.String. It seems this is a bug so currently we inserted a workaround below to avoid that.

fun TypeName.correctStringType() =
    if (this.toString() == "java.lang.String") ClassName("kotlin", "String") else this

[JakeWharton]

It's not really a bug. Kotlin uses a Java string when run on the JVM so that's what you see when using introspection APIs. Your workaround will incorrectly change non-Kotlin string types into Kotlin ones and it doesn't cover the hundreds of other JVM types that Kotlin ones map directly to.

[hotchemi]

Thx!

I understood the rationale and let me elaborate the situation I encountered.
Think about generating some pieces of code with annotation processor(kapt) from below.

@MarkerAnnotation
fun hoge(arg: String) {
  // do something
}

Then we expect such code is generated with KotlinPoet:

fun hogeWithKapt(arg: String) {
  // arg type is expected to be kotlin.String
}

But now we've got the code like:

fun hogeWithKapt(arg: java.lang.String) {
  // IDE flags warning about argument to use kotlin.String
}

Sorry I forgot to mention but our workaround is just for our library, so we don't have to consider any side effect about it(I didn't intend to add it to KotlinPoet for sure).

Anyway I realized it's kind of counter-intuitive because it means KotlinPoet user always take care about replacing java String to kotlin String. I don't come up with good way to deal with but I just reported as feedback 🙇

Please close this issue if it's not fit for your plan.

[JakeWharton]

We might be able to pull the Kotlin type information from the metadata annotations automatically.

[swankjesse]

Note that the two types aren’t interchangeable.

This compiles:

    val a : java.lang.String = java.lang.String("a")
    a.getBytes("UTF-8")

This doesn’t.

    val b : kotlin.String = "b"
    b.getBytes("UTF-8")

[tschuchortdev]

The same thing happens with kotlin.collections.List which turns into java.util.List thus breaking the generated code. Is there a solution for this yet?

@JakeWharton getting the type information from metadata is indeed possible. See which correctly returns `kotlin`.`collections`.`List`<`S`> for the parameter I tested.

[richardwrq]

I also have this problem, here is my solution.
I have defined an extension method for Element

/**
 *if null return, This means that this type is consistent with the kotlin type.
 */
private fun Element.javaToKotlinType(): ClassName? {
        val className = JavaToKotlinClassMap.INSTANCE.mapJavaToKotlin(FqName(this.asType().asTypeName().toString()))?.asSingleFqName()?.asString()
        return if (className == null) {
            null
        } else {
            ClassName.bestGuess(className)
        }
    }

you need to import kotlin-reflect

[tschuchortdev]

@richardwrq it doesn't seem to be working for me with generic classes like List. The type is correctly resolved, but instead of importing kotlin.collections.List it imports kotlin.collections.List<S> which of course doesn't work as well as the java version java.util.List leading to a conflicting import. I'm tempted to just remove all <.*?> from imports in the resulting file, but that's somewhat of a hack.
What types did you test this with?

[RiccardoM]

@JakeWharton Any news on this one?

[jdemeulenaere]

I'm interested in this as well :)

[zigzago]

@richardwrq @tschuchortdev This code is ok also for generic classes like List:

fun Element.javaToKotlinType(): TypeName =
        asType().asTypeName().javaToKotlinType()

 fun TypeName.javaToKotlinType(): TypeName {
        return if (this is ParameterizedTypeName) {
            ParameterizedTypeName.get(
                rawType.javaToKotlinType() as ClassName,
                *typeArguments.map { it.javaToKotlinType() }.toTypedArray()
            )
        } else {
            val className =
                JavaToKotlinClassMap.INSTANCE.mapJavaToKotlin(FqName(toString()))
                    ?.asSingleFqName()?.asString()

            return if (className == null) {
                this
            } else {
                ClassName.bestGuess(className)
            }
        }
    }
``

[RiccardoM]

@zigzago Works like a charm! Thank you

[StefMa]

@zigzago
This solution don't work with multiple generics:
List<List<Boolean>>
Will result in:
<List<out java.util.List<out Boolean>>>
(First List and last Boolean is correct. But second List is not correct).

I debugged a little bit and seems that the second List is not of type ParameterizedTypeName.

Additionally the solution uses internal APIs

import kotlin.reflect.jvm.internal.impl.name.FqName
import kotlin.reflect.jvm.internal.impl.platform.JavaToKotlinClassMap

which I don't want to rely on...

[zigzago]

@StefMa this is strange, because List<List<Boolean>> is ok on my side (tested with annotated kotlin class, not java class).

Anyway my solution is a workaround. As mentioned above, kotlin metadata should be used to really fix the issue.

[burningfireplace]

@zigzago
you can combine your workaround with the extension from #330 to make the type nullable if necessary. But there are still issues with this workaround:

• nullable type parameters become non-nullable
• you can't use mutable collections (MutableList for example)

Anyway, thanks for this solution!

@StefMa
List<List<Boolean>> works fine for me. I tried it with the return type of a method (getter method for a field / kotlin property)

[epool]

after 1.0.0-RC1 the above answer will need some changes in order to work, these lines to be specific.

ParameterizedTypeName.get(rawType.javaToKotlinType() as ClassName, *typeArguments.map { it.javaToKotlinType() }.toTypedArray())

====>

import com.squareup.kotlinpoet.ParameterizedTypeName.Companion.parameterizedBy
...
(rawType.javaToKotlinType() as ClassName).parameterizedBy(*typeArguments.map { it.javaToKotlinType() }.toTypedArray())

this was suggested on #424 .

[juanchosaravia]

In version 1.3.0-rc-190 JavaToKotlinClassMap is not located anymore in the same package.
Moved from
import kotlin.reflect.jvm.internal.impl.platform.JavaToKotlinClassMap
to
import kotlin.reflect.jvm.internal.impl.builtins.jvm.JavaToKotlinClassMap

[Erdenian]

Api has changed a bit (use parameterizedBy instead of ParameterizedTypeName.get):

private fun TypeName.javaToKotlinType(): TypeName = if (this is ParameterizedTypeName) {
    (rawType.javaToKotlinType() as ClassName).parameterizedBy(
        *typeArguments.map { it.javaToKotlinType() }.toTypedArray()
    )
} else {
    val className = JavaToKotlinClassMap.INSTANCE
        .mapJavaToKotlin(FqName(toString()))?.asSingleFqName()?.asString()
    if (className == null) this
    else ClassName.bestGuess(className)
}

[TAGC]

For me it's just JavaToKotlinClassMap.mapJavaToKotlin instead of JavaToKotlinClassMap.INSTANCE.mapJavaToKotlin

[dimsuz]

I'd like to note that in light of this "bug", FunSpec.overriding() is nearly impossible to use in my project.
Once the method being overrided contains String or List or any other type with kotlin substitute, the generated code fails to compile.

God bless the opensource nature of this library and kotlin's functions extensions: I was able to copy paste overriding() implementation in my project as a FunSpec extension making it apply the above workarounds inside. But I would really like if this was somehow resolved by this library (although I understand why it's not).

[AhmedMourad0]

Is the JavaToKotlinClassMap class still accessible? i can't seem to find it.

[JakeWharton]

A map is not the correct nor accurate solution to this problem since it's not a 1:1 mapping. Multiple distinct Kotlin types are represented as the same JVM type and it's only exacerbated by inline classes. Parsing the metadata is the only way forward.

[AhmedMourad0]

I see, any plans to implement this in the near future or is it staying in the IceBox?

[tschuchortdev]

Hasn't the metadata stuff already been merged?

[AhmedMourad0]

I'm using v1.4.1 and i still have this problem happening, so unless if i'm missing something, i don't think so.

[ZacSweers]

The metadata artifacts are separate and were released in 1.4.0. There are full READMEs in their artifacts on the repo.

It’s important to understand that metadata annotations are only present on classes. Not individual functions, types, properties, parameters, or anywhere else.

In short: cases like TypeMirror.asTypeName() or Type.asTypeName() will never work as intended for Kotlin compiler intrinsic types. APIs like this simply cannot look at individual types in isolation and have enough context to understand them. You must have access to the appropriate metadata that describes them in context. There are no silver bullets here, this is how Kotlin works.

You can, however, derive the metadata context from navigable elements like methods, fields, etc. You simply traverse up its hierarchy to the first enclosing class, which will have the needed metadata annotation on it. Then you can connect that parsed information back to your source element and connect the dots to deduce the correct type. This is exactly how the snippet I provided above for overriding works, and it works with the 1.4.0 metadata release.

There is possibly an argument to be made that either kotlinpoet-metadata artifacts should live directly in the main KotlinPoet artifact, or the existing problematic APIs should be deprecated and moved into the kotlinpoet-metadata artifact where they can ask for the needed classpath information. At this point I think that’s a separate issue.

[ywwynm]

@StefMa @zigzago List<List<String>> is converted to kotlin.collections.List<out java.util.List<java.lang.String>> at my side (kotlin version: 1.3.50&1.3.60). And I find the reason: the first List is ParameterizedTypeName so that it can be handled properly by javaToKotlinType(). However, the second List is actually WildcardTypeName (note the out) so that its converted type is just guessed by ClassName.

Interestingly, if we have a Map<List<Int>, List<Int>>, it will be converted to kotlin.collections.Map<kotlin.collections.List<kotlin.Int>, out java.util.List<java.lang.Integer>>>, which means the first List is regarded as ParameterizedTypeName correctly, but at the same time the second is WildcardTypeName.

As a result, we can improve javaToKotlinType() for WildcardTypeName case like this:

  fun TypeName.javaToKotlinType(): TypeName {
    return when (this) {
      is ParameterizedTypeName -> {
        (rawType.javaToKotlinType() as ClassName).parameterizedBy(*(typeArguments.map { it.javaToKotlinType() }.toTypedArray()))
      }
      is WildcardTypeName -> {
        outTypes[0].javaToKotlinType()
      }
      else -> {
        val className = JavaToKotlinClassMap.INSTANCE.mapJavaToKotlin(FqName(toString()))?.asSingleFqName()?.asString()
        return if (className == null) {
          this
        } else {
          ClassName.bestGuess(className)
        }
      }
    }
  }

And now it should work while I didn't test it thoroughly.

[KirillVolkov]

`
private fun TypeName.javaToKotlinType(): TypeName {

    return when (this) {
        is ParameterizedTypeName -> {
            (rawType.javaToKotlinType() as ClassName).parameterizedBy(
                *typeArguments.map {
                    it.javaToKotlinType()
                }.toTypedArray()
            )
        }
        is WildcardTypeName -> {
            val type =
                if (inTypes.isNotEmpty()) WildcardTypeName.consumerOf(inTypes[0].javaToKotlinType())
                else WildcardTypeName.producerOf(outTypes[0].javaToKotlinType())

            diagnostic(type.toString())
            type
        }

        else -> {
            val className = JavaToKotlinClassMap
                .mapJavaToKotlin(FqName(toString()))?.asSingleFqName()?.asString()
            if (className == null) {
                this
            } else {
                ClassName.bestGuess(className)
            }
        }
    }
}

`

[JohnOberhauser]

Is there a "correct" way to handle this yet?

[ZacSweers]

Yes, use the metadata APIs.

[JohnOberhauser]

@ZacSweers are there any examples on how to use that?

[ZacSweers]

Did you look at the readmes on both metadata artifacts?

[aouledissa]

Is there any resource/example one could follow to use the metadata APIs? I can't rely on the documentation of the metadata artefact as it's not clear at all.

[ZacSweers]

@andromedcodes there are READMEs in their modules. If those aren't sufficient, PR suggested improvements.

[aouledissa]

@ZacSweers do you mean this? https://github.com/square/kotlinpoet/tree/master/kotlinpoet-metadata

[ZacSweers]

https://github.com/square/kotlinpoet/blob/master/kotlinpoet-metadata/README.md

[aouledissa]

@ZacSweers Thanks. I have a question if I may. I have this annotation

@Retention(AnnotationRetention.SOURCE)
@Target(AnnotationTarget.CLASS)
annotation class Destination(
        val name: String = "",
        val args: Array<Arg> = []
)

@Retention(AnnotationRetention.SOURCE)
@Target(AnnotationTarget.VALUE_PARAMETER)
annotation class Arg(
        val name: String,
        val type: KClass<*>,
        val nullable: Boolean = false
)

as you can see args is an annotation but is used as an array of values in Destination
In the Arg annotation I pass a KClass<*> so i can generate later using kotlinpoet the parameter needed for this class. I have the same problem as above and I am stuck with the Type mirror giving me the java.lang.String instead of kotlin String class.
Any ideas how to solve this? or at least if you can point me to the right direction if I am doing something wrong.
PS: I can post the code if you need more in-depth look
Many thanks

[ZacSweers]

Please ask usage questions on stackoverflow. Given the general availability of metadata support, I'm going to close this issue out.

[aouledissa]

For any future reader here is a very interesting talk about metadata @JohnOberhauser
https://www.youtube.com/watch?v=uHPti6Z02tI

[obenabde]

Hey @ZacSweers I'm trying to use the overriding implementation provided above, but the "jvmMethodSignature" method in:

val jvmSignature = element.jvmMethodSignature(types)

can't be found anywhere. Has this been deleted? If or otherwise, is it possible to get an updated version of the code above? (also the test in the snippet uses the old signature, it needs newly added third parameter.)

[bhargavms]

Are there any documentation for kotlinpet metadata anywhere?

[ZacSweers]

Are there any documentation for kotlinpet metadata anywhere?

Did you look at their docs on the project site?

https://square.github.io/kotlinpoet/kotlinpoet_metadata/

https://square.github.io/kotlinpoet/kotlinpoet_metadata_specs/

[nima-flipp]

Are there any documentation for kotlinpet metadata anywhere?

Did you look at their docs on the project site?

https://square.github.io/kotlinpoet/kotlinpoet_metadata/

https://square.github.io/kotlinpoet/kotlinpoet_metadata_specs/

FYI both doc links are broken

[richardwrq]

您的来信已收到。谢谢

[Egorand]

@CasualGitEnjoyer the artifacts have been renamed, which invalidated the old URLs. Here's an up-to-date link for interop-kotlinx-metadata docs: https://square.github.io/kotlinpoet/interop-kotlinx-metadata/.