Fix compatibility on Android platforms below Android API 28 #543

overcat · 2023-09-21T12:10:16Z

Before merging this PR, we need to merge stellar/xdrgen#170 first.

After merging this PR, if Java 8+ API desugaring support is not introduced, this SDK can run on platforms >= Android API 24, which is our minimum supported version.
If you need to run on lower versions of the platform, you need to introduce Java 8+ API desugaring support in your project. According to my testing, after introducing it, it can run on platforms >= Android API 21. I have not tested on lower versions of the platform; perhaps it can run.

Change:

No longer use java.util.Base64, replace it with the code copied from commons-codec.
Add more tests on Android platform, adjust API level to 23

overcat · 2023-09-21T16:44:41Z

We need to discuss the minimum Android SDK version we should support before deciding which solution to use.

sreuland · 2023-09-21T19:53:44Z

We need to discuss the minimum Android SDK version we should support before deciding which solution to use.

@quietbits , @mollykarcher , do you have any insights or preferences on min version of android SDK?

overcat · 2023-09-22T03:14:35Z

The minSDK of the current Lobstr client is 23.

overcat · 2023-09-22T07:39:41Z

Android API Levels Cumulative usage: https://apilevels.com/

I tend to set the minimum supported version to 24, according to the information on the website above, which is enough to cover over 96% of devices. Additionally, many features of JDK 8 also require API 24+, on API 23, you can't even use java.util.Optional.

update: By introducing Java 8+ API desugaring support, we can use java.util.Optional.

sreuland · 2023-09-22T17:10:41Z

Android API Levels Cumulative usage: https://apilevels.com/

I tend to set the minimum supported version to 24, according to the information on the website above, which is enough to cover over 96% of devices. Additionally, many features of JDK 8 also require API 24+, on API 23, you can't even use java.util.Optional.

update: By introducing Java 8+ API desugaring support, we can use java.util.Optional.

Ok, thanks for recommendation on API 24, sounds good from a practical standpoint and 'in-the-field' insight.

sreuland · 2023-09-22T20:14:07Z

@overcat , it's great to see how your are adapting the sdk to work further in parallel on the android platform or native jdk. After review, one aspect stands out, this is making low level changes in the sdk which impact all usages but it is only relevant/needed for the android use case.

wanted to suggest a different design option to consider for allowing the sdk to scale out separately for native java or android specifics with a standard jdk mechanism, SPI(service provider interface) pattern to enable downstream apps to plug in different runtime implementations such as for Base64.

Here's a sketch of SPI classes to add in java sdk for Base64 provider:

package org.stellar.sdk;

public interface Base64 {
    private static Base64 base64provider = new JDKBase64Provider();
    static {
        ServiceLoader<SdkProvider> loader = ServiceLoader.load(SdkProvider.class);
        Iterator<SdKProvider> it = loader.iterator();
        while (it.hasNext()) {
            SdkProvider provider = it.next();
            base64Provider = provider.create();
        }
    }
   
    byte[] encode(byte[] data);
    byte[] decode(String data);
    String encodeToString(byte[] data);

    static Base64 getInstance() {
       return base64provider;
    }
}

public class JDKBase64Provider implements Base64 {
    byte[] encode(byte[] data) {
        // use java.util.Base64
    }

    byte[] decode(String data) {
        // use java.util.Base64
    }

    String encodeToString(byte[] data) {
         // use java.util.Base64
    }
}

package org.stellar.sdk.spi;

public interface SdkProvider {
    org.stellar.sdk.Base64 create();
}

So, in xdrgen, the code gen for static usage of Base64 would change slightly to Base64.getInstance().

And in the android project, can inject it's own SPI provider for different Base64 implementation:

package org.my.android.app;

public class AndroidSDKProvider implements SdkProvider {
    @Override
    public Base64 create() {
        return new AndroidBase64Provider();
    }
}

public class AndroidBase64Provider() {
 byte[] encode(byte[] data) {
        // use ApacheCodec
    }

    byte[] decode(String data) {
        // use ApacheCodec
    }

    String encodeToString(byte[] data) {
         // use ApacheCodec
    }
}

create file in android project src/main/resources/META-INF/services/org.stellar.sdk.spi.SDKProvider with contents:

org.my.android.app.AndroidSDKProvider

sreuland · 2023-09-22T20:35:28Z

@tamirms , @mollykarcher , @lijamie98 , @Ifropc would also appreciate any feedback related to this dual compatibility of native and android within single sdk - #543 (comment)

src/main/java/org/stellar/sdk/ApacheCodec.java

Ifropc · 2023-09-22T21:06:40Z

I agree that using provider would be a preferred approach. In fact, we had something similar in the wallet SDK in the past

overcat · 2023-09-23T02:40:41Z

Hi @sreuland and @Ifropc, SPI is a good proposal, but considering only the native Java platform and Android platform, I'm wondering if it's worth introducing it. This will make the SDK not plug-and-play, and users will need to introduce additional configurations.

overcat · 2023-09-23T02:43:18Z

I agree that using provider would be a preferred approach. In fact, we had something similar in the wallet SDK in the past

java.util.Base requires API 26 or higher, and the documentation here states that it requires API 23+.

Ifropc · 2023-09-23T05:03:33Z

Developers would need to manually change base64 implementation for old android devices and by default java native implementation is used.
Just wondering, do you see any issues with that?

overcat · 2023-09-23T14:21:07Z

Developers would need to manually change base64 implementation for old android devices and by default java native implementation is used. Just wondering, do you see any issues with that?

Using SPI can certainly meet the requirements. However, from the perspective of Android developers using this SDK, I think they may prefer the current implementation(?) Because they don't need to write their own provider.

overcat · 2023-09-24T08:22:00Z

I added an SPI implementation in the new PR(overcat-deprecation#26). If everyone prefers it, I will merge it into this PR.

let-it-snow · 2023-09-25T11:26:44Z

@overcat

I think they may prefer the current implementation(?) Because they don't need to write their own provider.

Sounds convenient and more appropriate than some additional configurations. At least for now.

sreuland · 2023-09-25T17:08:47Z

For android SPI usage, we could take a small extra leap, making it more seamless for devs by creating a new AndroidSdkSPIProvider java project which publishes to central/jitpack repos, a jar artifact with the SPI clasess and META-INF file:

package org.my.android.app;

public class AndroidSDKProvider implements SdkProvider {
    ...
}

public class AndroidBase64Provider() {
    ...
}

src/main/resources/META-INF/services/org.stellar.sdk.spi.SDKProvider

Android app imports AndroidSdkProvider as dependendcy implementation("com.github.stellar.:android-sdk-spi:0.0.1") , when that jar/lib is present in classpath, it wil triggers the jvm SPI to auto-load this for the SPI and override the built-in native jdk SPI usage, the additional step required by Android apps in this case, is to just include the new SPI dependency artifact in their android app, shouldn't require any additional coding. This is a common 'discovery' approach seen elsewhere in java ecosystem, such as by Spring Boot which detects optional 'impl' libs in classpath and auto-configs those into the final runtime.

overcat · 2023-09-26T07:02:31Z

Hi @sreuland, can you review this series of PRs? I have created a new repo, and once it is approved, I will transfer it to the gh stellar account.

refactor(java): Replace java.util.Base64 with org.stellar.sdk.Base64 to ensure compatibility on lower version Android platforms. stellar/xdrgen#170
Fix compatibility on Android platforms below Android API 28 #543
https://github.com/overcat/java-stellar-sdk-android-spi/pull/1 (new repo)

sreuland · 2023-09-26T22:01:04Z

android_test/app/src/main/java/org/stellar/javastellarsdkdemoapp/AndroidSdkProvider.kt

@@ -0,0 +1,24 @@
+package org.stellar.javastellarsdkdemoapp


once https://github.com/overcat/java-stellar-sdk-android-spi is published to jitpack, this can go away correct, and change this example to have dep on that instead?

[edit] oh, actually, i see it's already published to jitpack as a snapshot!

https://jitpack.io/#overcat/java-stellar-sdk-android-spi/-SNAPSHOT

sreuland · 2023-09-26T22:09:55Z

@overcat , https://github.com/overcat/java-stellar-sdk-android-spi/pull/1 looks good, I can't add myself as reviewer, but left approval comment. We'll get the setup going for that new repo on stellar in parallel, what's nice is that it's already published a snapshot to jitpack!

https://jitpack.io/#overcat/java-stellar-sdk-android-spi/-SNAPSHOT

so, can use that reference for dependencies in android test projects in the interim if that sounds ok?

sreuland

great work, thanks for consideration of the spi suggestion, looks great!

sreuland · 2023-09-26T22:35:03Z

src/main/java/org/stellar/sdk/ApacheCodec.java

this can/should be deleted now correct? since the andoid spi impl deals with these specifics now.

This includes the implementation of Base32, so it cannot be removed, but we can consider adding commons-codec back and then moving this copy to java-stellar-sdk-android-spi. Old versions of the Android platform include outdated commons-codec, it does not contain the Base32 module, see https://stackoverflow.com/questions/9126567/method-not-found-using-digestutils-in-android/29833101#29833101

that sounds like a worthwhile refactor, avoids the copy existing in the native jvm path, and that code copy then is really limited to just android apps at sdk level <= 25, which need to use the sdk provider impl override correct?

Yes, but it will affect users with API level <= 27. They need to introduce Android SPI.

ok, then should the docs on the sdk provider be consistent on that version, it looks version 26 is called out there as the compatibility line:

https://github.com/stellar/java-stellar-sdk/pull/543/files#diff-599240c9c2246349b86dff80f90edc454829c0a0a2228aa61c496f9333d154c1R20

I added it here: https://github.com/stellar/java-stellar-sdk/pull/543/files#diff-599240c9c2246349b86dff80f90edc454829c0a0a2228aa61c496f9333d154c1R36-R38

overcat · 2023-09-27T08:11:09Z

@overcat , overcat/java-stellar-sdk-android-spi#1 looks good, I can't add myself as reviewer, but left approval comment. We'll get the setup going for that new repo on stellar in parallel, what's nice is that it's already published a snapshot to jitpack!

https://jitpack.io/#overcat/java-stellar-sdk-android-spi/-SNAPSHOT

so, can use that reference for dependencies in android test projects in the interim if that sounds ok?

I don't intend to do this because it will result in circular dependencies. The java-stellar-sdk-android-spi depends on java-stellar-sdk, and the java-stellar-sdk project should not depend on java-stellar-sdk-android-spi. Additionally, java-stellar-sdk-android-spi has independent tests in its repo.

sreuland · 2023-09-27T17:53:21Z

I don't intend to do this because it will result in circular dependencies. The java-stellar-sdk-android-spi depends on java-stellar-sdk, and the java-stellar-sdk project should not depend on java-stellar-sdk-android-spi. Additionally, java-stellar-sdk-android-spi has independent tests in its repo.

ok, yeah, that's fine, it was one other additional 'leap' we could have done as part of the SPI pattern by putting the org.stellar.sdk.spi package in it's own project/lib like java-stellar-sdk-spi, which meant another repo as well, but I think this is adequate to carry the spi in the root project, and avoid the dependency cycle from test scenarios as you have.

# Conflicts: # CHANGELOG.md # src/main/java/org/stellar/sdk/StrKey.java

… Android tests.

overcat added 2 commits September 21, 2023 20:07

Remove commons-codec:commons-codec:1.16.0 and add a base32 copy.

98c48d4

update changelog

d7721d6

overcat requested a review from sreuland September 21, 2023 12:31

Improve Android testing.

ca81119

overcat changed the title ~~Remove commons-codec:commons-codec:1.16.0 and add a base32 copy.~~ [Dont Merge] Remove commons-codec:commons-codec:1.16.0 and add a base32 copy. Sep 21, 2023

overcat added 4 commits September 22, 2023 12:29

Copy base64

cf5c407

Update XDR

f033c29

Merge remote-tracking branch 'origin/issue/542' into issue/542

72f2951

empty commit, trigger a pipeline

8d40fd0

set min sdk to 24

8bdee59

overcat changed the title ~~[Dont Merge] Remove commons-codec:commons-codec:1.16.0 and add a base32 copy.~~ [Dont Merge] Remove commons-codec:commons-codec:1.16.0 and add a base32/base64 copy. Sep 22, 2023

overcat mentioned this pull request Sep 22, 2023

refactor(java): Replace java.util.Base64 with org.stellar.sdk.Base64 to ensure compatibility on lower version Android platforms. stellar/xdrgen#170

Merged

overcat changed the title ~~[Dont Merge] Remove commons-codec:commons-codec:1.16.0 and add a base32/base64 copy.~~ Remove commons-codec:commons-codec:1.16.0 and add a base32/base64 copy. Sep 22, 2023

overcat added 2 commits September 22, 2023 18:05

check base64 string

367da02

Merge remote-tracking branch 'origin/issue/542' into issue/542

5cb5dff

overcat changed the title ~~Remove commons-codec:commons-codec:1.16.0 and add a base32/base64 copy.~~ Fix compatibility on Android platforms below Android API 28 Sep 22, 2023

Ifropc reviewed Sep 22, 2023

View reviewed changes

src/main/java/org/stellar/sdk/ApacheCodec.java Outdated Show resolved Hide resolved

add License

4ed6821

fix

f8a3373

add test

d88f5de

overcat added 3 commits September 26, 2023 09:11

SPI

3ca599d

add docs

d1e86e6

add docs

ad4ed93

sreuland reviewed Sep 26, 2023

View reviewed changes

sreuland approved these changes Sep 26, 2023

View reviewed changes

sreuland reviewed Sep 26, 2023

View reviewed changes

base32 spi

503a82c

overcat requested a review from sreuland September 27, 2023 17:16

overcat mentioned this pull request Sep 27, 2023

Fix the issue of javadocJar not including documentation & Publish sourcesJar to the GitHub Release page. #539

Merged

overcat added 3 commits September 28, 2023 10:08

Merge branch 'master' into issue/542

0a513a6

# Conflicts: # CHANGELOG.md # src/main/java/org/stellar/sdk/StrKey.java

update XDRGEN_COMMIT

6eded56

Update test-deploy.yml to use the wretry action for retrying unstable…

d423f57

… Android tests.

overcat merged commit 63c7531 into lightsail-network:master Sep 28, 2023
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix compatibility on Android platforms below Android API 28 #543

Fix compatibility on Android platforms below Android API 28 #543

overcat commented Sep 21, 2023 •

edited

Loading

overcat commented Sep 21, 2023

sreuland commented Sep 21, 2023

overcat commented Sep 22, 2023

overcat commented Sep 22, 2023 •

edited

Loading

sreuland commented Sep 22, 2023

sreuland commented Sep 22, 2023

sreuland commented Sep 22, 2023

Ifropc commented Sep 22, 2023

overcat commented Sep 23, 2023

overcat commented Sep 23, 2023

Ifropc commented Sep 23, 2023

overcat commented Sep 23, 2023

overcat commented Sep 24, 2023 •

edited

Loading

let-it-snow commented Sep 25, 2023

sreuland commented Sep 25, 2023

overcat commented Sep 26, 2023

sreuland Sep 26, 2023 •

edited

Loading

sreuland commented Sep 26, 2023

sreuland left a comment

sreuland Sep 26, 2023

overcat Sep 26, 2023

sreuland Sep 26, 2023

overcat Sep 26, 2023 •

edited

Loading

sreuland Sep 26, 2023

overcat Sep 27, 2023

overcat commented Sep 27, 2023

sreuland commented Sep 27, 2023

Fix compatibility on Android platforms below Android API 28 #543

Fix compatibility on Android platforms below Android API 28 #543

Conversation

overcat commented Sep 21, 2023 • edited Loading

Change:

overcat commented Sep 21, 2023

sreuland commented Sep 21, 2023

overcat commented Sep 22, 2023

overcat commented Sep 22, 2023 • edited Loading

sreuland commented Sep 22, 2023

sreuland commented Sep 22, 2023

sreuland commented Sep 22, 2023

Ifropc commented Sep 22, 2023

overcat commented Sep 23, 2023

overcat commented Sep 23, 2023

Ifropc commented Sep 23, 2023

overcat commented Sep 23, 2023

overcat commented Sep 24, 2023 • edited Loading

let-it-snow commented Sep 25, 2023

sreuland commented Sep 25, 2023

overcat commented Sep 26, 2023

sreuland Sep 26, 2023 • edited Loading

Choose a reason for hiding this comment

sreuland commented Sep 26, 2023

sreuland left a comment

Choose a reason for hiding this comment

sreuland Sep 26, 2023

Choose a reason for hiding this comment

overcat Sep 26, 2023

Choose a reason for hiding this comment

sreuland Sep 26, 2023

Choose a reason for hiding this comment

overcat Sep 26, 2023 • edited Loading

Choose a reason for hiding this comment

sreuland Sep 26, 2023

Choose a reason for hiding this comment

overcat Sep 27, 2023

Choose a reason for hiding this comment

overcat commented Sep 27, 2023

sreuland commented Sep 27, 2023

overcat commented Sep 21, 2023 •

edited

Loading

overcat commented Sep 22, 2023 •

edited

Loading

overcat commented Sep 24, 2023 •

edited

Loading

sreuland Sep 26, 2023 •

edited

Loading

overcat Sep 26, 2023 •

edited

Loading