AppImage: add needed VA-API libraries (Intel & AMD) #2429

gschintgen · 2024-04-16T19:44:41Z

Description

Currently VA-API encoding may/will fail when the AppImage is used. This seems to be because of a mismatch between the bundled libva.so and the system's /usr/lib/*/dri/*_drv_video.so driver libraries. ~~Since this is a rather important functionality the necessary libraries are now bundled in the AppImage itself in a relatively recent version.~~ This PR changes the AppImage build process in order to include a recent libva version. This should fix VA-API issues for Intel and AMD based systems. ~~I left out nouveau_drv_video.so since I'm somewhat skeptical of its encoding capabilities. (Please tell me if I'm wrong...)~~

I tested this PR on an older Intel-based laptop (Ubuntu 22.04, i965 driver) / on my main AMDGPU-based desktop. It's working fine for me.

Screenshot

Issues Fixed or Closed

See #2409 (but does not fix it since Flatpak is untouched)

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Dependency update (updates to dependencies)
Documentation update (changes to documentation)
Repository update (changes to repository files, e.g. .github/...)

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

I want maintainers to keep my branch updated

ReenigneArcher · 2024-04-16T20:42:29Z

.github/workflows/CI.yml

@@ -320,6 +320,8 @@ jobs:
        run: |
          # allow newer gcc
          sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
+          # use a recent Mesa version
+          sudo add-apt-repository ppa:kisak/kisak-mesa -y


I see the repo added, but what package is coming from here?

Thanks for asking. I added a comment giving a few more details. Here's the complete list as extracted from the CI build logs:

libwayland-server0:amd64 (1.20.0-1ubuntu0.1~kisak~f) libwayland-bin (1.20.0-1ubuntu0.1~kisak~f) libva2:amd64 (2.14.0-1~kisak1~f) libglapi-mesa:amd64 (24.0.5~kisak1~f) libvdpau1:amd64 (1.4-3build2~kisak~f) libllvm15:amd64 (1:15.0.7-10~kisak1~f) libwayland-egl1:amd64 (1.20.0-1ubuntu0.1~kisak~f) libvdpau-dev:amd64 (1.4-3build2~kisak~f) libdrm-common (2.4.120-1~kisak~f) libwayland-client0:amd64 (1.20.0-1ubuntu0.1~kisak~f) libdrm2:amd64 (2.4.120-1~kisak~f) libva-drm2:amd64 (2.14.0-1~kisak1~f) libwayland-cursor0:amd64 (1.20.0-1ubuntu0.1~kisak~f) libva-x11-2:amd64 (2.14.0-1~kisak1~f) libva-glx2:amd64 (2.14.0-1~kisak1~f) libdrm-amdgpu1:amd64 (2.4.120-1~kisak~f) libdrm-nouveau2:amd64 (2.4.120-1~kisak~f) libgbm1:amd64 (24.0.5~kisak1~f) libdrm-radeon1:amd64 (2.4.120-1~kisak~f) libdrm-intel1:amd64 (2.4.120-1~kisak~f) libgl1-mesa-dri:amd64 (24.0.5~kisak1~f) libwayland-dev:amd64 (1.20.0-1ubuntu0.1~kisak~f) libgbm-dev:amd64 (24.0.5~kisak1~f) libegl-mesa0:amd64 (24.0.5~kisak1~f) libva-wayland2:amd64 (2.14.0-1~kisak1~f) mesa-va-drivers:amd64 (24.0.5~kisak1~f) mesa-vdpau-drivers:amd64 (24.0.5~kisak1~f) libdrm-dev:amd64 (2.4.120-1~kisak~f) libglx-mesa0:amd64 (24.0.5~kisak1~f) libva-dev:amd64 (2.14.0-1~kisak1~f) va-driver-all:amd64 (2.14.0-1~kisak1~f) vdpau-driver-all:amd64 (1.4-3build2~kisak~f) libgl1-mesa-dev:amd64 (24.0.5~kisak1~f) libegl1-mesa-dev:amd64 (24.0.5~kisak1~f)

It's quite an update over 20.04's mesa; some testing can't hurt.

packaging/linux/AppImage/AppRun

codecov · 2024-04-16T20:57:39Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 6.80%. Comparing base (0a595dc) to head (7451cc0).

Additional details and impacted files

@@            Coverage Diff             @@
##           nightly   #2429      +/-   ##
==========================================
- Coverage     6.99%   6.80%   -0.20%     
==========================================
  Files           87      87              
  Lines        17679   17672       -7     
  Branches      8399    8399              
==========================================
- Hits          1237    1202      -35     
+ Misses       13867   13833      -34     
- Partials      2575    2637      +62

Flag	Coverage Δ
Linux	`5.34% <ø> (ø)`
Windows	`2.56% <ø> (ø)`
macOS-12	`?`
macOS-13	`7.91% <ø> (+0.01%)`	⬆️
macOS-14	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 32 files with indirect coverage changes

gschintgen · 2024-04-17T20:32:10Z

I'm marking the PR as draft as long as there are unresolved issues with AMD GPUs. (See #2409 (comment))

.github/workflows/CI.yml

cgutman

The way VA-API's backwards compatibility guarantees work is that the version of libva.so at runtime must be >= the version of libva that the driver (_drv_video.so) was compiled against. This is done via looking up a __vaDriverInit function that is decorated with the compile-time version number of VA-API.

For example, libva.so 2.19 wil look for __vaDriverInit_1_19(), then __vaDriverInit_1_18(), __vaDriverInit_1_17() and so on. So drivers built with any version equal or less than 2.19 will work, but a driver built for libva 2.20 (which only has __vaDriverInit_1_20()) will not work.

Due to this backwards compatibilty guarantee, essentially all we need to do for optimal compatibility is bundle whatever is the latest version of libva.so (currently 2.21) into the AppImage (this is also what Moonlight does). I don't think we should be bundling the _drv_video.so files because I think those have dependencies on Mesa and could break if we run them with a different version of Mesa than they were built as part of.

gschintgen · 2024-04-20T07:58:55Z

essentially all we need to do for optimal compatibility is bundle whatever is the latest version of libva.so (currently 2.21)

Thanks for your insightful and helpful comment! I've added changes to use the latest libva (from a PPA), but leaving out the *_drv_video.so. Let's see how it goes.

gschintgen · 2024-04-21T10:58:54Z

Oh well, the kisak-mesa PPA only has libva at version 2.14 instead of a more recent one. I guess for optimal support we'll have to build it ourselves. (That is if the latest versions of libva are compatible with a 4 year old distribution.)

I'll play around with it but it could take some time.

gschintgen · 2024-04-23T15:14:48Z

I force-pushed a new change overwriting the previous PR contents. This time we're building our own copy of the latest libva but we're using the host system's VA-API drivers. (Thanks cgutman )

On my own fork I've already tested the resulting AppImage and it is working fine on my system (cursory testing of remote desktop):

libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
[2024:04:23:16:54:30]: Warning: [hevc_vaapi @ 0x564a9a4d1380] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:04:23:16:54:30]: Info: 
[2024:04:23:16:54:30]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:04:23:16:54:30]: Info: 
[2024:04:23:16:54:30]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:04:23:16:54:30]: Info: Found HEVC encoder: hevc_vaapi [vaapi]

.github/workflows/CI.yml

gschintgen · 2024-04-29T10:36:18Z

I see that in the meantime the AppImage has been upgraded from Ubuntu 20.04 to 22.04. I don't think that change invalidates the logic behind this PR. I rebased to fix a minor merge conflict.

.github/workflows/CI.yml

gschintgen · 2024-05-05T15:27:03Z

Are there any merge blockers for this PR? I think for supported distributions this should work fine and fix the initial issue. (Which I can confirm on my own system.)

The CI failures seem to be independent of the PR. For example the Analyze (cpp, ubuntu-latest) fails due to this:

/root/cuda.run      100%[===================>]   4.04G  75.3MB/s    in 55s     
terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::copy_file: No space left on device: "./builds/nsight_systems/documentation/pdf/UserGuide.pdf", "/usr/nsight-systems-2022.4.2/documentation/pdf/UserGuide.pdf"

cuda is huge!?

ReenigneArcher · 2024-05-05T15:57:51Z

cuda is huge!?

Basically yea. GitHub modifies runners from time to time and it causes out of space failures. I tried to fix it in #2475 but it didn't work.

Libva nees to be at least as recent as the host system's VA-API hardware drivers. (*_drv_video.so)

ReenigneArcher reviewed Apr 16, 2024

View reviewed changes

ReenigneArcher added the pr:planned label Apr 16, 2024

ReenigneArcher added this to the v0.23.1 milestone Apr 16, 2024

gschintgen mentioned this pull request Apr 17, 2024

Linux VAAPI Doesn't Work With AppImage or Flatpak #2409

Closed

3 tasks

gschintgen marked this pull request as draft April 17, 2024 20:32

ReenigneArcher reviewed Apr 17, 2024

View reviewed changes

.github/workflows/CI.yml Outdated Show resolved Hide resolved

cgutman requested changes Apr 20, 2024

View reviewed changes

ReenigneArcher removed this from the v0.23.1 milestone Apr 21, 2024

gschintgen force-pushed the fix-appimage-vaapi branch from f17e564 to 888ddb8 Compare April 23, 2024 15:06

ReenigneArcher reviewed Apr 24, 2024

View reviewed changes

.github/workflows/CI.yml Outdated Show resolved Hide resolved

gschintgen force-pushed the fix-appimage-vaapi branch from 2717b25 to 8aa0890 Compare April 25, 2024 11:34

gschintgen marked this pull request as ready for review April 25, 2024 11:35

gschintgen force-pushed the fix-appimage-vaapi branch from 8aa0890 to 1d79214 Compare April 29, 2024 10:33

ReenigneArcher reviewed Apr 29, 2024

View reviewed changes

.github/workflows/CI.yml Outdated Show resolved Hide resolved

cgutman approved these changes May 13, 2024

View reviewed changes

gschintgen added 3 commits May 12, 2024 22:34

AppImage: build our own libva

8e7a1c2

Libva nees to be at least as recent as the host system's VA-API hardware drivers. (*_drv_video.so)

AppImage: use latest libva release instead of current git

976ba85

AppImage: make libva build options more explicit

7451cc0

ReenigneArcher force-pushed the fix-appimage-vaapi branch from 1909c1b to 7451cc0 Compare May 13, 2024 02:34

ReenigneArcher merged commit a5bbf36 into LizardByte:nightly May 13, 2024
51 of 53 checks passed

gschintgen deleted the fix-appimage-vaapi branch May 13, 2024 13:04

KuleRucket pushed a commit to KuleRucket/Sunshine that referenced this pull request Jun 6, 2024

AppImage: add needed VA-API libraries (Intel & AMD) (LizardByte#2429)

57b4a37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AppImage: add needed VA-API libraries (Intel & AMD) #2429

AppImage: add needed VA-API libraries (Intel & AMD) #2429

gschintgen commented Apr 16, 2024 •

edited

Loading

ReenigneArcher Apr 16, 2024

gschintgen Apr 17, 2024

codecov bot commented Apr 16, 2024 •

edited

Loading

gschintgen commented Apr 17, 2024

cgutman left a comment •

edited

Loading

gschintgen commented Apr 20, 2024

gschintgen commented Apr 21, 2024

gschintgen commented Apr 23, 2024

gschintgen commented Apr 29, 2024

gschintgen commented May 5, 2024

ReenigneArcher commented May 5, 2024

AppImage: add needed VA-API libraries (Intel & AMD) #2429

AppImage: add needed VA-API libraries (Intel & AMD) #2429

Conversation

gschintgen commented Apr 16, 2024 • edited Loading

Description

Screenshot

Issues Fixed or Closed

Type of Change

Checklist

Branch Updates

ReenigneArcher Apr 16, 2024

Choose a reason for hiding this comment

gschintgen Apr 17, 2024

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2024 • edited Loading

Codecov Report

gschintgen commented Apr 17, 2024

cgutman left a comment • edited Loading

Choose a reason for hiding this comment

gschintgen commented Apr 20, 2024

gschintgen commented Apr 21, 2024

gschintgen commented Apr 23, 2024

gschintgen commented Apr 29, 2024

gschintgen commented May 5, 2024

ReenigneArcher commented May 5, 2024

gschintgen commented Apr 16, 2024 •

edited

Loading

codecov bot commented Apr 16, 2024 •

edited

Loading

cgutman left a comment •

edited

Loading