Support for Kotlin Multiplatform targets #1

aSemy · 2023-05-31T10:57:50Z

Hi 👋

I'd like to be able to use this library in a Kotlin Multiplatform project. Would you consider updating the library to support JVM, JS, and Native Kotlin compilation targets?

For now I've migrated it manually. I'm including the code so you are welcome to use it.

If you would like some help with the Gradle config, I can provide a PR.

Summary of changes:

converted to a class
converted safeChars to class constructor parameter
replaced BitSet with BooleanArray
updated ByteArray operations to use Kotlin Multiplatform equivalents
added some assertions (based on Google PercentEscaper)

/*
 * Copyright 2001-2023 Geert Bevin (gbevin[remove] at uwyn dot com)
 * Copyright 2022-2023 Erik C. Thauvin (erik@thauvin.net)
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

// Adapted version of UrlEncoder.kt
// - converted to a class
// - converted safeChars to class constructor parameter
// - replaced BitSet with BooleanArray
// - updated ByteArray operations to use Kotlin Multiplatform equivalents
// - added some assertions (based on Google PercentEscaper)
// https://github.com/ethauvin/urlencoder/blob/34b69a7d1f3570aa056285253376ed7a7bde03d8/lib/src/main/kotlin/net/thauvin/erik/urlencoder/UrlEncoder.kt

package net.thauvin.erik.urlencoder

/**
 * Most defensive approach to URL encoding and decoding.
 *
 * - Rules determined by combining the unreserved character set from
 * [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986#page-13) with the percent-encode set from
 * [application/x-www-form-urlencoded](https://url.spec.whatwg.org/#application-x-www-form-urlencoded-percent-encode-set).
 *
 * - Both specs above support percent decoding of two hexadecimal digits to a binary octet, however their unreserved
 * set of characters differs and `application/x-www-form-urlencoded` adds conversion of space to `+`, which has the
 * potential to be misunderstood.
 *
 * - This library encodes with rules that will be decoded correctly in either case.
 *
 * @param safeChars a non-null string specifying additional safe characters for this escaper (the
 * ranges `0..9`, `a..z` and `A..Z` are always safe and should not be specified here)
 * @param plusForSpace `true` if ASCII space should be escaped to `+` rather than `%20`
 *
 * @author Geert Bevin (gbevin(remove) at uwyn dot com)
 * @author Erik C. Thauvin (erik@thauvin.net)
 **/
internal class UrlEncoder(
    safeChars: String,
    private val plusForSpace: Boolean,
) {

    // see https://www.rfc-editor.org/rfc/rfc3986#page-13
    // and https://url.spec.whatwg.org/#application-x-www-form-urlencoded-percent-encode-set
    private val unreservedChars = createUnreservedChars(safeChars)

    init {
        // Avoid any misunderstandings about the behavior of this escaper
        require(!safeChars.matches(".*[0-9A-Za-z].*".toRegex())) {
            "Alphanumeric characters are always 'safe' and should not be explicitly specified"
        }
        // Avoid ambiguous parameters. Safe characters are never modified so if
        // space is a safe character then setting plusForSpace is meaningless.
        require(!(plusForSpace && ' ' in safeChars)) {
            "plusForSpace cannot be specified when space is a 'safe' character"
        }
        require('%' !in safeChars) {
            "The '%' character cannot be specified as 'safe'"
        }
    }

    /**
     * Transforms a provided [String] into a new string, containing decoded URL characters in the UTF-8
     * encoding.
     */
    fun decode(
        source: String,
        plusToSpace: Boolean = plusForSpace,
    ): String {
        if (source.isEmpty()) return source

        val length = source.length
        val out = StringBuilder(length)
        var ch: Char
        var bytesBuffer: ByteArray? = null
        var bytesPos = 0
        var i = 0
        var started = false
        while (i < length) {
            ch = source[i]
            if (ch == '%') {
                if (!started) {
                    out.append(source, 0, i)
                    started = true
                }
                if (bytesBuffer == null) {
                    // the remaining characters divided by the length of the encoding format %xx, is the maximum number
                    // of bytes that can be extracted
                    bytesBuffer = ByteArray((length - i) / 3)
                }
                i++
                require(length >= i + 2) { "Incomplete trailing escape ($ch) pattern" }
                try {
                    val v = source.substring(i, i + 2).toInt(16)
                    require(v in 0..0xFF) { "Illegal escape value" }
                    bytesBuffer[bytesPos++] = v.toByte()
                    i += 2
                } catch (e: NumberFormatException) {
                    throw IllegalArgumentException("Illegal characters in escape sequence: $e.message", e)
                }
            } else {
                if (bytesBuffer != null) {
                    out.append(bytesBuffer.decodeToString(0, bytesPos))
                    started = true
                    bytesBuffer = null
                    bytesPos = 0
                }
                if (plusToSpace && ch == '+') {
                    if (!started) {
                        out.append(source, 0, i)
                        started = true
                    }
                    out.append(" ")
                } else if (started) {
                    out.append(ch)
                }
                i++
            }
        }

        if (bytesBuffer != null) {
            out.append(bytesBuffer.decodeToString(0, bytesPos))
        }

        return if (!started) source else out.toString()
    }

    /**
     * Transforms a provided [String] object into a new string, containing only valid URL
     * characters in the UTF-8 encoding.
     *
     * - Letters, numbers, unreserved (`_-!.'()*`) and allowed characters are left intact.
     */
    fun encode(
        source: String,
        spaceToPlus: Boolean = plusForSpace,
    ): String {
        if (source.isEmpty()) {
            return source
        }
        var out: StringBuilder? = null
        var ch: Char
        var i = 0
        while (i < source.length) {
            ch = source[i]
            if (ch.isUnreserved()) {
                out?.append(ch)
                i++
            } else {
                if (out == null) {
                    out = StringBuilder(source.length)
                    out.append(source, 0, i)
                }
                val cp = source.codePointAt(i)
                if (cp < 0x80) {
                    if (spaceToPlus && ch == ' ') {
                        out.append('+')
                    } else {
                        out.appendEncodedByte(cp)
                    }
                    i++
                } else if (Character.isBmpCodePoint(cp)) {
                    for (b in ch.toString().encodeToByteArray()) {
                        out.appendEncodedByte(b.toInt())
                    }
                    i++
                } else if (Character.isSupplementaryCodePoint(cp)) {
                    val high = Character.highSurrogateOf(cp)
                    val low = Character.lowSurrogateOf(cp)
                    for (b in charArrayOf(high, low).concatToString().encodeToByteArray()) {
                        out.appendEncodedByte(b.toInt())
                    }
                    i += 2
                }
            }
        }

        return out?.toString() ?: source
    }

    /**
     * see https://www.rfc-editor.org/rfc/rfc3986#page-13
     * and https://url.spec.whatwg.org/#application-x-www-form-urlencoded-percent-encode-set
     */
    private fun Char.isUnreserved(): Boolean = this <= 'z' && unreservedChars[code]

    companion object {

        private val hexDigits: CharArray = "0123456789ABCDEF".toCharArray()

        private fun StringBuilder.appendEncodedDigit(digit: Int) {
            append(hexDigits[digit and 0x0F])
        }

        private fun StringBuilder.appendEncodedByte(ch: Int) {
            append("%")
            appendEncodedDigit(ch shr 4)
            appendEncodedDigit(ch)
        }

        /**
         * Creates a [BooleanArray] with entries corresponding to the character values for
         * `0-9`, `A-Z`, `a-z` and those specified in [safeChars] set to `true`.
         *
         * The array is as small as is required to hold the given character information.
         */
        private fun createUnreservedChars(safeChars: String): BooleanArray {
            val safeCharArray = safeChars.toCharArray()
            val maxChar = safeCharArray.maxOf { it.code }.coerceAtLeast('z'.code)

            val unreservedChars = BooleanArray(maxChar + 1)

            unreservedChars['-'.code] = true
            unreservedChars['.'.code] = true
            unreservedChars['_'.code] = true
            for (c in '0'..'9') unreservedChars[c.code] = true
            for (c in 'A'..'Z') unreservedChars[c.code] = true
            for (c in 'a'..'z') unreservedChars[c.code] = true

            for (c in safeCharArray) unreservedChars[c.code] = true

            return unreservedChars
        }
    }
}

Character utils

// Based on https://github.com/cketti/kotlin-codepoints

/*
 * MIT License
 *
 * Copyright (c) 2023 cketti
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in all
 * copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 * SOFTWARE.
 */

/**
 * Kotlin Multiplatform equivalent for `java.lang.Character`
 */
internal object Character {

    /**
     * See https://www.tutorialspoint.com/java/lang/character_issupplementarycodepoint.htm
     *
     * Determines whether the specified character (Unicode code point) is in the supplementary character range.
     * The supplementary character range in the Unicode system falls in `U+10000` to `U+10FFFF`.
     *
     * The Unicode code points are divided into two categories:
     * Basic Multilingual Plane (BMP) code points and Supplementary code points.
     * BMP code points are present in the range U+0000 to U+FFFF.
     *
     * Whereas, supplementary characters are rare characters that are not represented using the original 16-bit Unicode.
     * For example, these type of characters are used in Chinese or Japanese scripts and hence, are required by the
     * applications used in these countries.
     *
     * @returns `true` if the specified code point falls in the range of supplementary code points
     * ([MIN_SUPPLEMENTARY_CODE_POINT] to [MAX_CODE_POINT], inclusive), `false` otherwise.
     */
    internal fun isSupplementaryCodePoint(codePoint: Int): Boolean =
        codePoint in MIN_SUPPLEMENTARY_CODE_POINT..MAX_CODE_POINT

    internal fun charCount(codePoint: Int): Int = if (codePoint <= MIN_SUPPLEMENTARY_CODE_POINT) 1 else 2

    internal fun toCodePoint(highSurrogate: Char, lowSurrogate: Char): Int =
        (highSurrogate.code shl 10) + lowSurrogate.code + SURROGATE_DECODE_OFFSET

    internal fun toChars(codePoint: Int): CharArray = when {
        isBmpCodePoint(codePoint) -> charArrayOf(codePoint.toChar())
        else                      -> charArrayOf(highSurrogateOf(codePoint), lowSurrogateOf(codePoint))
    }

    /** Basic Multilingual Plane (BMP) */
    internal fun isBmpCodePoint(codePoint: Int): Boolean = codePoint ushr 16 == 0

    internal fun highSurrogateOf(codePoint: Int): Char =
        ((codePoint ushr 10) + HIGH_SURROGATE_ENCODE_OFFSET.code).toChar()

    internal fun lowSurrogateOf(codePoint: Int): Char =
        ((codePoint and 0x3FF) + MIN_LOW_SURROGATE.code).toChar()

    private const val MIN_CODE_POINT: Int = 0x000000
    private const val MAX_CODE_POINT: Int = 0x10FFFF

    private const val MIN_SUPPLEMENTARY_CODE_POINT: Int = 0x10000

    private const val SURROGATE_DECODE_OFFSET: Int =
        MIN_SUPPLEMENTARY_CODE_POINT -
            (MIN_HIGH_SURROGATE.code shl 10) -
            MIN_LOW_SURROGATE.code

    private const val HIGH_SURROGATE_ENCODE_OFFSET: Char = MIN_HIGH_SURROGATE - (MIN_SUPPLEMENTARY_CODE_POINT ushr 10)

}



/**
 * Returns the Unicode code point at the specified index.
 *
 * The `index` parameter is the regular `CharSequence` index, i.e. the number of `Char`s from the start of the character
 * sequence.
 *
 * If the code point at the specified index is part of the Basic Multilingual Plane (BMP), its value can be represented
 * using a single `Char` and this method will behave exactly like [CharSequence.get].
 * Code points outside the BMP are encoded using a surrogate pair – a `Char` containing a value in the high surrogate
 * range followed by a `Char` containing a value in the low surrogate range. Together these two `Char`s encode a single
 * code point in one of the supplementary planes. This method will do the necessary decoding and return the value of
 * that single code point.
 *
 * In situations where surrogate characters are encountered that don't form a valid surrogate pair starting at `index`,
 * this method will return the surrogate code point itself, behaving like [CharSequence.get].
 *
 * If the `index` is out of bounds of this character sequence, this method throws an [IndexOutOfBoundsException].
 *
 * To iterate over all code points in a character sequence the index has to be adjusted depending on the value of the
 * returned code point. Use [CodePoints.charCount] for this.
 *
 * ```kotlin
 * // Text containing code points outside the BMP (encoded as a surrogate pairs)
 * val text = "\uD83E\uDD95\uD83E\uDD96"
 *
 * var index = 0
 * while (index < text.length) {
 *     val codePoint = text.codePointAt(index)
 *     // Do something with codePoint
 *
 *     index += CodePoints.charCount(codePoint)
 * }
 * ```
 */
internal fun CharSequence.codePointAt(index: Int): Int {
    if (index !in indices) throw IndexOutOfBoundsException("index $index was not in range $indices")

    val firstChar = this[index]
    if (firstChar.isHighSurrogate()) {
        val nextChar = getOrNull(index + 1)
        if (nextChar?.isLowSurrogate() == true) {
            return Character.toCodePoint(firstChar, nextChar)
        }
    }

    return firstChar.code
}

The text was updated successfully, but these errors were encountered:

ethauvin · 2023-05-31T16:46:48Z

HI @aSemy,

Please submit a PR with your code changes and Gradle config. I'll review it and incorporate it.

Thanks!

E.

aSemy · 2023-05-31T17:11:29Z

@ethauvin Sure thing, I'll make a PR with my changes above and another with the Gradle config

ethauvin · 2023-05-31T19:38:04Z

@aSemy Feel free to change/add to the README file too

aSemy · 2023-05-31T19:41:48Z

What do you think about supporting the CLI app?

It will be difficult to support a multiplatform CLI (although not impossible). But that's not necessary - it's perfectly possible to keep the JVM CLI application while migrating the backing code to Kotlin Multiplatform.

I think the best way to support this would be to split up the library into two subprojects:

:lib - the Kotlin Multiplatform library (essentially the code in my initial comment)
:app - a Kotlin/JVM CLI app.

In order to support backwards compatibility the UrlEncoder could remain in :app, while the encode/decode functions could be moved to a new class - named UrlEncoderSupport? UrlEncoderLib?

I'd like this change because I wanted to use UrlEncoder in a library, and I have no need for a main function. And also it's useful to be able to create a single instance and re-use it rather than continuously passing in the same arguments.

ethauvin · 2023-05-31T19:46:03Z

@aSemy I like that too. As long as we keep backward compatibility, I’m good with it.

aSemy · 2023-06-04T19:48:57Z

There are two more points to think about:

When moving to Kotlin Multiplatform, the artifact with have the target suffixed on the end of it.

Now for Gradle that doesn't matter, because Gradle can automagically determine the correct usage. However, Maven isn't as clever and so Maven users will have to add a dependency using
```
net.thauvin.erik:urlencoder-jvm:1.3.0
```
It's possible to try and hack around to avoid this, but I think it's risky and requires maintenance.

On the other hand, while requiring Mavan users add -jvm is a bit annoying, it's a small change, and Maven users are probably used to it from other Kotlin Multiplatform projects.

So what do you think? Update the README or try and update the POM?
Currently the tests use JUnit, which is JVM only. There are two options:
1. Don't update the tests. Only the JVM will be tested.
2. Update the tests to use a multiplatform-compatible test library. kotlin-test is the most similar to JUnit. Kotest is another option, but it would require a much larger refactoring.

ethauvin · 2023-06-04T21:53:00Z

So what do you think? Update the README or try and update the POM?

Just update the documentation.

Currently the tests use JUnit, which is JVM only. There are two options:

Don't update the tests. Only the JVM will be tested.

Update the tests to use a multiplatform-compatible test library. kotlin-test is the most similar to JUnit. Kotest is another option, but it would require a much larger refactoring.

I'm okay with kotlin-test, I've used it often.

Thanks.

ethauvin · 2023-06-05T19:00:06Z

@aSemy As I suspected, the sonar issue fixed itself when I merged the branches. I guess you were right, most likely a permission issue.

ethauvin · 2023-06-05T19:49:37Z

@aSemy So the sonar task is working, but the koverXmlReport task is always being skipped. Any ideas what's going on there? Without the report, sonar won't work properly.

aSemy · 2023-06-05T20:03:22Z

Good to hear that Sonar is still working!

Next step is to migrate the project to a Kotlin/Multiplatform structure, while still keeping everything as JVM. After that, I can update the main to Multiplatform, and after that the tests.

Regarding Kover: I had a quick look and I could see Kover was running for :app and :lib, but it's skipping the root project because there aren't any tests (which makes sense). Maybe Sonar needs the report to be aggregated? I can try setting up an aggregated Kover XML report.

aSemy · 2023-06-05T20:54:18Z

Ahh yes, it is an issue with the Sonar token not being available.

When I look in the logs for the action run against the PR, the token isn't available:

https://github.com/ethauvin/urlencoder/actions/runs/5181538553/jobs/9337181042?pr=5#step:7:12

While on an action run against master branch, it is available:

https://github.com/ethauvin/urlencoder/actions/runs/5180967788/jobs/9335838718#step:7:12

ethauvin · 2023-06-05T21:09:27Z

@aSemy Last PR is working like a charm. Good job!

aSemy · 2023-06-05T21:30:03Z

great!

ethauvin · 2023-06-06T21:34:41Z

Currently the tests use JUnit, which is JVM only. There are two options:

Don't update the tests. Only the JVM will be tested.

Update the tests to use a multiplatform-compatible test library. kotlin-test is the most similar to JUnit. Kotest is another option, but it would require a much larger refactoring.

Have you figured out how to use parameters in test in kotlin.test? Looks like Kotest supports that directly.

ethauvin · 2023-06-06T21:40:03Z

Have you figured out how to use parameters in test in kotlin.test? Looks like Kotest supports that directly.

Looking that tests, a simple array loop would work with minimal changes. Do you want me to refactor the tests with Kotlin.testor you have something else in mind?

ethauvin · 2023-06-07T05:44:43Z

@aSemy I converted the tests to kotlin.test.

ethauvin · 2023-06-07T06:23:24Z

@aSemy looks like the app tests are not processed by kover.

aSemy · 2023-06-07T12:57:09Z

Great work! Looks good. I think kotlin-test is a better idea than Kotest. It's more simple, but it's more stable, and it's more similar to JUnit.

Have you figured out how to use parameters in test in kotlin.test? Looks like Kotest supports that directly.

There's no built-in support for parameterized tests. There are two options: either iterate over the test data in a single test (like you've done), or create a function that is called multiple times by @Test functions.

@Test
fun `Main Decode 1`() = mainDecord("a test &" to "a%20test%20%26")
fun `Main Decode 2`() = mainDecord("!abcdefghijklmnopqrstuvwxyz%%ABCDEFGHIJKLMNOPQRSTUVQXYZ0123456789-_.~=" to                 "%21abcdefghijklmnopqrstuvwxyz%25%25ABCDEFGHIJKLMNOPQRSTUVQXYZ0123456789-_.%7E%3D")

private fun mainDecode(m: Pair<String, String>) {
  val result: UrlEncoder.MainResult = processMain(arrayOf("-d", m.second))
  assertEquals(m.first, result.output)
  assertEquals(0, result.status, "processMain(-d ${m.second}).status")
}

The benefit is that a single failing test will be easier to track down, because each test case is test directly. The downside is that it's a little more code.

aSemy · 2023-06-07T13:14:52Z

I just checked out master and ran ./gradlew check, and I can see the XML report for :app is generated

ethauvin · 2023-06-07T15:38:45Z

The benefit is that a single failing test will be easier to track down, because each test case is test directly. The downside is that it's a little more code.

I think the way I did it is just fine; the parameters are included in the failure message which makes it really easy to track down too. Not much different from the way it worked with junit.

ethauvin · 2023-06-07T15:40:02Z

I just checked out master and ran ./gradlew check, and I can see the XML report for :app is generated

Ah, good catch. I guess I need to let sonar know about it.

ethauvin · 2023-08-04T16:16:04Z

@aSemy The PR has been merged.

I can't seem to be able to publish the snapshot(s) at this point:

* What went wrong:
Could not determine the dependencies of task ':urlencoder-app:dokkaJavadoc'.
> Empty collection can't be reduced.

And I'm a little confused as to how you envisioned the publishing? The KVM lib and also the app? Let me know. I can easily set up an action to do test the snapshot publishing, if you need it.

Also, could you look at the README and make sure it is accurately reflecting the multi-platform options, etc.

Let me know if I can do anything to help.

Thanks,

E.

aSemy · 2023-08-06T09:01:35Z

Good to see that there's some progress, even if it's not working perfectly :)

> Empty collection can't be reduced.

That looks like this issue Kotlin/dokka#3063. In this project we can resolve it by adding additional Kotlin targets (JS, and native targets). I'll make a new PR for this.

Additionally, I'd recommend removing the generated Dokka from the Javadoc JAR. I don't think putting generated documentation Javadoc JAR is very useful, because I doubt many users are going to download, unzip, and view the generated docs. The sources JAR is much more useful, as this will be picked up by IDEs and contains the exact same documentation.

Also, could you look at the README and make sure it is accurately reflecting the multi-platform options, etc.

Sure, I'll take a look. The instructions for Maven users should be updated, because they'll be required to add a -jvm suffix.

ethauvin · 2023-08-06T17:23:30Z

Additionally, I'd recommend removing the generated Dokka from the Javadoc JAR. I don't think putting generated documentation Javadoc JAR is very useful, because I doubt many users are going to download, unzip, and view the generated docs. The sources JAR is much more useful, as this will be picked up by IDEs and contains the exact same documentation.

@aSemy I don't think that's a good idea. Maven Central, some IDEs, etc. are expecting the Javadoc jar to be there.

I've merged the PRs, but haven't had a chance to test much yet, although ./gradlew check worked for me on Linux.

aSemy · 2023-08-07T10:05:33Z

Maven Central requires a Javadoc JAR is present, but it doesn't have any requirements on the content! I've used an empty Javadoc JAR in in a few of my projects. But it doesn't hurt to include some content in the JAR, so if you want to keep it then go for it :)

I've made #9 to enable more OSes in the GitHub workflow, so that will help with testing.

ethauvin · 2023-08-07T16:39:00Z

I've made #9 to enable more OSes in the GitHub workflow, so that will help with testing.

Thanks. I've fixed the workflow failing with Windows and JDK 17/20 with e5cb0bd

I still can't publish a snapshot:

❯ gradle publish
executing gradlew instead of gradle
Type-safe project accessors is an incubating feature.
> Task :urlencoder-lib:dokkaJavadoc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':urlencoder-lib:dokkaJavadoc'.
> Could not resolve all files for configuration ':urlencoder-lib:dokkaJavadocPlugin'.
   > Resolved 'com.soywiz.korlibs.korte:korte-jvm:2.7.0' which is not part of the dependency lock state
   > Resolved 'org.jetbrains.dokka:javadoc-plugin:1.8.20' which is not part of the dependency lock state
   > Resolved 'org.jetbrains.dokka:kotlin-as-java-plugin:1.8.20' which is not part of the dependency lock state

Any ideas?

ethauvin · 2023-09-03T21:42:09Z

Hey @aSemy any ideas on the above?

aSemy · 2023-09-04T10:04:01Z

hey @ethauvin, I haven't looked into it but I guess it's a problem with Dokka and the Gradle lock files. I haven't used this combination, so I don't an answer to hand. It would either require some investigation, or you could ditch the lock file & Dokka requirements and attempt to add them back in later, after multiplatform support is stable?

ethauvin · 2023-09-04T19:21:35Z

hey @ethauvin, I haven't looked into it but I guess it's a problem with Dokka and the Gradle lock files. I haven't used this combination, so I don't an answer to hand. It would either require some investigation, or you could ditch the lock file & Dokka requirements and attempt to add them back in later, after multiplatform support is stable?

@aSemy I tried that, but I was getting errors with the JS stuff, etc. Do you mind taking a look at it?

aSemy · 2023-09-04T19:52:36Z

I can take a look, but what happened? What errors did you see?

ethauvin · 2023-09-04T20:05:39Z

I can take a look, but what happened? What errors did you see?

I removed the lock files and I got this when running check:

Could not determine the dependencies of task ':kotlinNpmInstall'.
> Could not resolve all dependencies for configuration ':urlencoder-app:jsNpmAggregated'.
   > Locking strict mode: Configuration ':urlencoder-app:jsNpmAggregated' is locked but does not have lock state.

Which got me quite puzzled.

aSemy · 2023-09-04T20:11:45Z

did you also remove the config that enables the lockfiles?

urlencoder/buildSrc/src/main/kotlin/buildsrc/conventions/base.gradle.kts

Lines 20 to 36 in 32e2149

    
           dependencyLocking { 
        
               lockMode.set(LockMode.STRICT) 
        
               lockAllConfigurations() 
        
           } 
        
           tasks.register("resolveAndLockAllDependencies") { 
        
               // https://docs.gradle.org/current/userguide/dependency_locking.html#ex-resolving-all-configurations 
        
               group = "dependencies" 
        
               notCompatibleWithConfigurationCache("Filters configurations at execution time") 
        
               val resolvableConfigurations = configurations.matching { it.isCanBeResolved } 
        
               doFirst { 
        
                   require(gradle.startParameter.isWriteDependencyLocks) { "$path must be run from the command line with the `--write-locks` flag" } 
        
               } 
        
               doLast { 
        
                   resolvableConfigurations.forEach { it.resolve() } 
        
               } 
        
           }

urlencoder/buildSrc/build.gradle.kts

Lines 13 to 22 in 32e2149

    
           dependencyLocking { 
        
               lockMode.set(LockMode.STRICT) 
        
           } 
        
           configurations { 
        
               // only lock compile & runtime to prevent error: 
        
               // Locking strict mode: Configuration ':buildSrc:testCompileClasspath' is locked but does not have lock state. 
        
               compileClasspath { resolutionStrategy.activateDependencyLocking() } 
        
               runtimeClasspath { resolutionStrategy.activateDependencyLocking() } 
        
           }

ethauvin · 2023-09-04T21:42:28Z

did you also remove the config that enables the lockfiles?

@aSeemy. I didn't, that helped. I also found a way to make dokka only generate docs for the jvm, but now I'm getting:

Execution failed for task ':urlencoder-lib:generateMetadataFileForLinuxX64Publication'.
> java.io.FileNotFoundException: /home/erik/dev/kotlin/urlencoder/urlencoder-lib/build/classes/kotlin/linuxX64/main/klib/urlencoder-lib.klib (No such file or directory)

when trying to publish.

aSemy · 2023-09-05T08:07:54Z

Haha the problems don't stop coming!

Fortunately I recognise this one - it's because there are no sources in commonMain https://youtrack.jetbrains.com/issue/KT-52344/

There are two options

create a couple of empty files
- urlencoder-lib/src/commonMain/kotlin/empty.kt
- urlencoder-app/src/commonMain/kotlin/empty.kt
convert the project to multiplatform

ethauvin · 2023-09-05T08:33:40Z

2. convert the project to multiplatform

@aSemy What does that entitle?

aSemy · 2023-09-05T09:15:30Z

convert the project to multiplatform

@aSemy What does that entitle?

Basically the changes I shared right at the top. In short: replace JVM only types and functions with Kotlin Multiplatform equivalents.

I think it's easier to look at the code differences, so I made a PR: #10

ethauvin · 2023-09-05T21:14:57Z

@aSemy Success! I was finally able to publish a snapshot

I think it's publishing too many things. Things like urlencoder-lib or urlencoded-app. Thoughts?

aSemy · 2023-09-06T06:38:31Z

Nice! I can see these artifacts, which look right to me. The app and lib were broken up, and there's a specific artifact for each Kotlin target.

(The number of variants doesn't make a difference to Gradle users - thanks to some magic Gradle will be able to select the correct variant from the base url-encoder artifact. Maven users will have to specify the -jvm suffix though.)

urlencoder-app-jvm
urlencoder-app
urlencoder-lib-js
urlencoder-lib-jvm
urlencoder-lib-linuxx64
urlencoder-lib-mingwx64
urlencoder-lib

So actually it looks like some artifacts are missing - there aren't any macOS variants. Unfortunately publishing macOS variants requires a macOS machine. Do you have a Mac to hand? If not, we can set up publishing using a GitHub Action, since they have free Mac runners.

ethauvin · 2023-09-06T07:09:38Z

Nice! I can see these artifacts, which look right to me. The app and lib were broken up, and there's a specific artifact for each Kotlin target.

(The number of variants doesn't make a difference to Gradle users - thanks to some magic Gradle will be able to select the correct variant from the base url-encoder artifact. Maven users will have to specify the -jvm suffix though.)

Okay, that makes sense.

urlencoder-app-jvm
urlencoder-app
urlencoder-lib-js
urlencoder-lib-jvm
urlencoder-lib-linuxx64
urlencoder-lib-mingwx64
urlencoder-lib	
So actually it looks like some artifacts are missing - there aren't any macOS variants. Unfortunately publishing macOS variants requires a macOS machine. Do you have a Mac to hand? If not, we can set up publishing using a GitHub Action, since they have free Mac runners.

Yeah, we'll have to do that. I don't use MacOS products anymore. I'll work on it.

Thanks!

ethauvin · 2023-09-06T21:40:47Z

So actually it looks like some artifacts are missing - there aren't any macOS variants. Unfortunately publishing macOS variants requires a macOS machine. Do you have a Mac to hand? If not, we can set up publishing using a GitHub Action, since they have free Mac runners.

@aSemy I pushed the publish action. Could you please double-check that it is creating everything it should in the repo? You should be able to manually run it, if needed.

Thanks!

ethauvin · 2023-09-06T22:38:56Z

@aSemy I'm thinking of changing the package id/group to net.thauvin.erik.urlencoder so everything is in one directory in Maven. Thoughts?

p.s. I ended up doing it, everything is there now:

https://oss.sonatype.org/content/repositories/snapshots/net/thauvin/erik/urlencoder/

aSemy · 2023-09-13T09:59:11Z

Looks like the snapshot version works!

https://github.com/krzema12/snakeyaml-engine-kmp/pull/92/files

Congrats on the Kotlin Multiplatform library!

There might be an issue where UrlEncoder requires the latest Kotlin version - you might want to consider setting the Kotlin language level to 1.5 (here, I think), so UrlEncoder can be used more widely.

I'm thinking of changing the package id/group to net.thauvin.erik.urlencoder so everything is in one directory in Maven. Thoughts?

It makes sense to me 👍, it certainly looks more tidy.

ethauvin · 2023-09-13T20:27:37Z

There might be an issue where UrlEncoder requires the latest Kotlin version - you might want to consider setting the Kotlin language level to 1.5 (here, I think), so UrlEncoder can be used more widely.

I set it to 1.6, as 1.5 is deprecated since January.

Thanks for all your work on this; I don't think I could have done it on my own.

### Summary Migrate UrlEncoder util to the now converted & published UrlEncoder library ethauvin/urlencoder#1 Co-authored-by: Piotr Krzeminski <git@krzeminski.it>

ethauvin added the enhancement New feature or request label May 31, 2023

ethauvin assigned ethauvin and aSemy May 31, 2023

aSemy mentioned this issue May 31, 2023

tidy up .gitignore and .idea/ files #2

Merged

aSemy mentioned this issue May 31, 2023

split up app/lib #4

Merged

aSemy mentioned this issue Aug 6, 2023

Update README with Kotlin Multiplatform info, add Contributing guide with lockfile instructions #8

Merged

aSemy mentioned this issue Sep 5, 2023

Convert urlencoder lib to Kotlin Multiplatform #10

Merged

aSemy mentioned this issue Sep 13, 2023

use UrlEncoder dependency krzema12/snakeyaml-engine-kmp#92

Merged

ethauvin closed this as completed Sep 22, 2023

Support for Kotlin Multiplatform targets #1

Support for Kotlin Multiplatform targets #1

Comments

aSemy commented May 31, 2023 • edited Loading

Character utils

ethauvin commented May 31, 2023

aSemy commented May 31, 2023

ethauvin commented May 31, 2023

aSemy commented May 31, 2023

ethauvin commented May 31, 2023

aSemy commented Jun 4, 2023 • edited Loading

ethauvin commented Jun 4, 2023 • edited Loading

ethauvin commented Jun 5, 2023

ethauvin commented Jun 5, 2023

aSemy commented Jun 5, 2023

aSemy commented Jun 5, 2023

ethauvin commented Jun 5, 2023

aSemy commented Jun 5, 2023

ethauvin commented Jun 6, 2023

ethauvin commented Jun 6, 2023

ethauvin commented Jun 7, 2023

ethauvin commented Jun 7, 2023

aSemy commented Jun 7, 2023

aSemy commented Jun 7, 2023 • edited Loading

ethauvin commented Jun 7, 2023

ethauvin commented Jun 7, 2023

ethauvin commented Aug 4, 2023

aSemy commented Aug 6, 2023

ethauvin commented Aug 6, 2023 • edited Loading

aSemy commented Aug 7, 2023 • edited Loading

ethauvin commented Aug 7, 2023

ethauvin commented Sep 3, 2023

aSemy commented Sep 4, 2023

ethauvin commented Sep 4, 2023

aSemy commented Sep 4, 2023

ethauvin commented Sep 4, 2023

aSemy commented Sep 4, 2023

ethauvin commented Sep 4, 2023

aSemy commented Sep 5, 2023

ethauvin commented Sep 5, 2023

aSemy commented Sep 5, 2023 • edited Loading

ethauvin commented Sep 5, 2023

aSemy commented Sep 6, 2023 • edited Loading

ethauvin commented Sep 6, 2023

ethauvin commented Sep 6, 2023 • edited Loading

ethauvin commented Sep 6, 2023 • edited Loading

aSemy commented Sep 13, 2023 • edited Loading

ethauvin commented Sep 13, 2023

aSemy commented May 31, 2023 •

edited

Loading

aSemy commented Jun 4, 2023 •

edited

Loading

ethauvin commented Jun 4, 2023 •

edited

Loading

aSemy commented Jun 7, 2023 •

edited

Loading

ethauvin commented Aug 6, 2023 •

edited

Loading

aSemy commented Aug 7, 2023 •

edited

Loading

aSemy commented Sep 5, 2023 •

edited

Loading

aSemy commented Sep 6, 2023 •

edited

Loading

ethauvin commented Sep 6, 2023 •

edited

Loading

ethauvin commented Sep 6, 2023 •

edited

Loading

aSemy commented Sep 13, 2023 •

edited

Loading