Remove libthai from the excludelist and improve excludelist formatting

[Korne127]

This Pull Request removes libthai.so.0 from the excludelist. The original reason why libthai has been added to it is an issue that occurs when libpango is not bundled but libthai is (see #538). But since then, libpango has been removed from the excludelist in 2022. This means that the original reason why libthai has been excluded is no longer relevant. Therefore it can be removed from the excludelist. This fixes #538.

Additionally, this improves the excludelist formatting. There have been different places scattered across the excludelist in which libraries have been excluded or not excluded due to other libraries (like libthai / libpango). This brings them all (with all information and reasoning) in one block to prevent issues like this in the future. Additionally, it adds missing information out of #528 to this block.

[probonopd]

Thanks @Korne127 for your thoughtful contribution. 👍

[Korne127]

Thank you so much!! :D

[Korne127]

This fixes numerous issues in other projects that use AppImages for packaging, including:

• [bug] appimage does not contain libthai.so tauri-apps/tauri#4930
• AppImage missing libthai cinnyapp/cinny-desktop#59
• [Bug] missing libthai.so.0 in AppImage build lencx/ChatGPT#312
• Can't run AppImage under NixOS aptakube/aptakube#78
• AppImage doesn't launch without libthai LizardByte/Sunshine#1792
• Appimage does not include libary libthai modrinth/code#900
• AppImage doesn't work when libthai isn't installed gitbutlerapp/gitbutler#2740
• Appimage is broken: libthai.so.0 not found SlimeVR/SlimeVR-Server#819
• AppImage lacks libthai.so.0 h3poteto/fedistar#654
• libthai.so.0 shared library error when launching v1.2.4 appimage lumehq/lume#88
• Missing libthai in appimage open-goal/launcher#368
• error while loading shared libraries: libthai.so.0: cannot open shared object file: No such file or directory spacebarchat/client#208
• AppImage doesn't launch on Gentoo YARC-Official/YARC-Launcher#33
• v1.12.2 appimage wants libthai webosbrew/dev-manager-desktop#189
• Flatpak, AppImage franciscoBSalgueiro/en-croissant#148
• AppImage libthai.so and pixbuf loaders not found (SOLVED) xournalpp/xournalpp#4660
• AppImage seemes not to be static cjcliffe/CubicSDR#852
• libthai.so.0 - missing from appImage tw93/Pake#531
• appimage missing libthai xemu-project/xemu#1530
• appImage: "/tmp/.mount_DarktaTmcr11/AppRun.wrapped: error while loading shared libraries: libthai.so.0: cannot open shared object file: No such file or directory" darktable-org/darktable#16871
• Library inclusion request Xpl0itU/WiiUDownloader#79
• AppImage not starting. Library missing on Slackware ( libthai not found ) xournalpp/xournalpp#4124
• Error installing Query qarmin/czkawka#375
• libthai.so.0 not supplied linuxdeploy/linuxdeploy-plugin-gtk#23
• xLights-2021.40-x86_64.AppImage missing libthai library xLightsSequencer/xLights#3038
• AppImage release depend on "Golang IDE by JetBrains" through libthai.so.0 audacity/audacity#5846

[Norbiros]

For the fix to take effect in e.g. Tauri apps, we'll have to wait for a new release of linuxdeploy, as they need to update their exclude list (generate-excludelist.sh).

[probonopd]

Before we are all to confident with this solution, I suggest that we test running an application bundled in this way on a target system that has a never version of those libraries installed locally... to be sure that we haven't broken anything.

Are the applications listed above all using pkg2appimage? That's not the recommended way to make official (first-party) AppImages. pkg2appimage is really just a hack for end users who must convert deb packages because the original application authors are not providing AppImages. Tools like go-appimage's appimagetool would be more suited for that imho.

[Korne127]

Before we are all to confident with this solution, I suggest that we test running an application bundled in this way on a target system that has a never version of those libraries installed locally... to be sure that we haven't broken anything.

I tried this with --library /lib/x86_64-linux-gnu/libthai.so.0 in the linuxdeploy command (on a friend's Gentoo system) and it worked. I strongly assume removing libthai.so.0 from the excludelist will provide the exact same result, but I will test it again.

Are the applications listed above all using pkg2appimage?

No. But linuxdeploy (which is afaik the usual / preferred way to build AppImages) (and possibly also other tools) also uses this pkg2appimage excludelist. Therefore, when building an AppImage via linuxdeploy, it was susceptible for this issue.

Many of these issues weren't even directly using any AppImage tool but were using tauri, which itself uses linuxdeploy.
The issues were just all issues that were linked to the original issue (or another linked issue, I went through them recursively).
But I'll now search for other issues like this as well and add them, since many are probably not mentioned.

[probonopd]

Ah right, thanks for the clarification @Korne127.

I don't know how linuxdeploy does it, but it might be that for the changes to take effect, those projects that use the excludelist from this repository would have to be updated and/or rebuilt.

[Korne127]

I don't know how linuxdeploy does it

See this link - essentially it just gets the latest file on the Github master.
probonopd/linuxdeployqt#274 explains how the file is used:
A version of the file is bundled with linuxdeploy (and linuxdeployqt). The version is generated when building linuxdeploy (in CMakeLists.txt) and contains the then-current excludelist.
But every time when you try to use the command linuxdeploy and package something, the tool tries to automatically update the excludelist and get the new internet via Github.

@Norbiros This means that you don't need to wait for a linuxdeploy update; as long as your system has internet access while packaging, the file should automatically be updated and this issue be fixed.
The projects themselves just need to release a new version.
Edit: Not only did this just apply to linuxdeployqt, I completely misunderstood the solution; every version of it only contains one static excludelist.

[Korne127]

I went through all issues I could find and added the additional ones I found to the list (and commented under some issues that have partly or possibly been fixed). :D

[probonopd]

linuxdeployqt != linuxdeploy

[Korne127]

I know, but they both handle this in the same way (the way I just described).

Edit: I was wrong. I checked both, but I must have made a mistake.

[Norbiros]

The projects themselves just need to release a new version.

I compiled Tauri app on my own after the PR was merged, but I still encountered the libthai is missing error.

@Norbiros This means that you don't need to wait for a linuxdeploy update; as long as your system has internet access while packaging, the file should automatically be updated and this issue be fixed.

Oh, sorry for the confusion. Tauri downloads linuxdeploy from their binary releases (source code). To address this issue, they need to compile the newest version of linux deploy (so CMakeList.txt would execute generate-excludelist.sh) and update artifacts.

[Korne127]

You were right, I tested it as well with linuxdeploy and among the output I got

Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libthai.so.0

I relooked into the code and I'm sorry, I made a mistake. Only linuxdeployqt tries to update the excludelist with every command by calling generate-exclude list.sh. (Edit: I was wrong again and misunderstood the issue. None of these projects do that (yet)). linuxdeploy itself has only the feature of getting the new excludelist with every release, so we need a new linuxdeploy release in order for this to be resolved on most projects.

[Korne127]

I created an issue at linuxdeploy/linuxdeploy#286 to address this as not trying to update the excludelist at build time creates a big dependency on new linuxdeploy updates. I'll try to make a PR for this later today.

@probonopd Are you in personal contact with @TheAssassin? It seems like they're the only person who could merge a corresponding PR / create a new linuxdeploy release and fix this for downstream developers (which is a bit risky for such a hugely depended upon project tbh).

[probonopd]

Indeed, @TheAssassin is the person in charge of linuxdeploy, but I happen to know that he is very busy these days.

[Korne127]

Thanks for the info

[TheAssassin]

I created an issue at linuxdeploy/linuxdeploy#286 to address this as not trying to update the excludelist at build time creates a big dependency on new linuxdeploy updates. I'll try to make a PR for this later today.

Same old story, including things at build time is annoying and triggers an avalanche of rebuilds everywhere.