F-Droid can't build

[licaon-kter]

Checklist

• I am able to reproduce the bug with the latest version given here: CLICK THIS LINK.
• I made sure that there are no existing issues - open or closed - which I could contribute my information to.
• I have read the FAQ and my problem isn't listed.
• I have taken the time to fill in all the required details. I understand that the bug report will be dismissed otherwise.
• This issue contains only one bug.
• I have read and understood the contribution guidelines.

Affected version

0.27.3

Steps to reproduce the bug

ref: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/17014

was the APK built from d298a12 ?

Expected behavior

No response

Actual behavior

No response

Screenshots/Screen recordings

No response

Logs

diff log: newp1000.log

Affected Android/Custom ROM version

No response

Affected device model

No response

Additional information

No response

[TobiGr]

was the APK built from d298a12 ?

Yes.

The problem is that some Enums are not desugared correctly: diff1.html.zip. We probably need to update the desugaring libs, any other ideas?

[obfusk]

Have you tried simply building with OpenJDK 21? The rbtlog rebuild 4 days ago was perfectly fine, no sign of any desugaring problems.

Edit: RB now confirmed by two builders: https://apt.izzysoft.de/fdroid/index/apk/org.schabi.newpipe

[Stypox]

I created a CI to build the APK in a known environment, so we won't be having such issues again: #11765. The unsigned APK built by https://github.com/Stypox/NewPipe/actions/runs/12096387804 using JDK 17 is exactly equal to the one built by fdroid build in https://gitlab.com/fdroid/fdroiddata/-/jobs/8482164620/artifacts/file/unsigned/org.schabi.newpipe_1000.apk . I guess we can either change the release APK we distribute for v0.27.3, or make a new release with the new APK.

[obfusk]

I guess we can either change the release APK we distribute for v0.27.3, or make a new release with the new APK.

Please don't replace existing APKs or tags, it's considered bad practice. As @IzzySoft always puts it: "One shouldn't replace what has already be distributed", which is why we always ask for a "maintenance release" instead. There should never be two non-identical versions (with different hashes) of the same release, even if there are no functional differences. That only creates confusion, breaks anything using the old hash for validation, and in case of RB would also invalidate any existing RB confirmations from rebuilders.

[Stypox]

That makes a lot of sense, we've published v0.27.4 now. Thank you for your patience once again!

[IzzySoft]

@Stypox so your APKs are again using JDK-21 starting with v0.27.4? Then I'd run a manual update on my builder to avoid a double-run (delaying RB result publication) later.

[Stypox]

No, they will be using JDK 17, that's what you asked for right? The original APK for 0.27.3 was built by me with the JDK 21 shipped with Android Studio, and it was not reproducible with respect to the one built by the F-Droid bot (which afaik uses JDK 17 (?)). Building with temurin JDK 17 according to the workflow in https://github.com/TeamNewPipe/NewPipe/blob/dev/.github/workflows/build-release-apk.yml leads to an unsigned APK for 0.27.3 which is exactly the same as the one built by the F-Droid bot.
Let us know which version to use, and we can stick to it from now on. Sorry for the confusion, I shouldn't have built the APK with Android Studio's JDK in the first place.

[obfusk]

• Indeed F-Droid builds with OpenJDK 17 by default; they can also easily switch to OpenJDK 11, as both are provided by Debian bookworm.
• They can also use OpenJDK 21, but that requires getting it from Debian trixie (testing), which is not ideal for RB as that's a moving target; other releases like 22 are not LTS and thus very much not recommended at all (e.g. they will almost certainly not ship with Debian trixie).
• Our independent rbtlog builders (used by e.g. IzzyOnDroid) are more flexible and not tied to a single Debian release so we can easily build using e.g. ubuntu:jammy, which is why our builders could trivially switch to OpenJDK 21 to confirm RB for v0.27.3.
• One of our rebuilders already verified v0.27.4 as well, after switching back to OpenJDK 17 🎉
• So F-Droid should indeed now be able to build v0.27.4 without needing any changes on their end.
• Using OpenJDK 21 is usually fine, but switching from 17 to 21 means a change to the (re)build recipe is needed to avoid breaking RB as the APK will not be identical unless both builds use the same version, as you've seen :)
• As various other apps already use OpenJDK 21 (and our rebuild for v0.27.3 was fine after switching to that) I'm surprised F-Droid didn't even try using that 🤷‍♀️

[IzzySoft]

No, I didn't ask for JDK-17 – obfusk and I reported our builders had no issues building it with JDK-21. Our builders are quite flexible, and can easily use JDKs 11, 17 or 21 where needed (we stick to those "stable versions", though, and e.g. do not use 18 or 22) – or switch between building on Debian bookworm/bullseye and Ubuntu jammy/noble as well.

@licaon-kter do you really need JDK-17, or could you switch to JDK-21 as well? If I'm not mistaken, you're glued to bookworm currently which does not have JDK-21 – but I occasionally see you "frankensteining" in DEBs from other distributions: 45 apps at F-Droid are using JDK-21 if I counted correctly. We can easily go back to 17 if you need that (and if it's OK with the NewPipe team, which one day might need to upgrade to JDK-21). We should just all agree on what it should be; switching back-and-forth would cause "constant trouble" for all involved.

TL;DR: At IzzyOnDroid, we're fine with either 17 or 21. You just need to tell us which it should be from now on. So it's rather between you @Stypox and @licaon-kter to agree on a JDK version.

Building with temurin JDK 17 according to the workflow

That is a very good point, @Stypox: that workflow is public, and folks use it for orientation (I do look for such workflows when setting up new recipes). So maybe sticking with 17 for the time being is the best choice. I'll prepare our recipe for that then ASAP, before the next cycle here kicks in (runs every 24h).

[obfusk]

FYI: the other CI workflow uses 21, not 17:

NewPipe/.github/workflows/ci.yml

Line 53 in 3847b32

java-version: 21

NewPipe/.github/workflows/ci.yml

Line 127 in 3847b32

java-version: 21

[IzzySoft]

One of our rebuilders already verified v0.27.4 as well, after switching back to OpenJDK 17 🎉

make that two 🥳

[licaon-kter]

We usually see this type of differences between 17 and 21: https://f-droid.org/docs/Reproducible_Builds/#mismatched-toolchains so this diff did not lean in that direction

As said above, we can use 21 too @Stypox up to you and the team what you use, we'll amend the recipe as needed

/LE: verified and merged in https://gitlab.com/fdroid/fdroiddata/-/merge_requests/17177

[Stypox]

Great, thank you! Ok then, let's agree on using JDK 21 from now on @licaon-kter @IzzySoft. I will change our CI and our documentation to make sure we stick to that version in the future.

[TobiGr]

So I guess we need to add something like this to the metadata of our next f-droid builds (taken from the metadata of another app):

    sudo:
      - echo "deb http://deb.debian.org/debian trixie main" > /etc/apt/sources.list.d/trixie.list
      - apt-get update
      - apt-get install -y -t trixie openjdk-21-jdk-headless
      - update-java-alternatives -a

[licaon-kter]

@TobiGr yup