Enable vulkan for ffmpeg (Windows, Linux, Mac) #1460

streamingdv · 2024-01-12T13:43:10Z

This is is an attempt to enable vulkan for ffmpeg.
Unfortuently builds for Android are not passing because of following error

vulkan requested but not found

It seems the Android builds require more configuration in order to find the Vulkan SDK. Other builds seem to work fine and it seems to be enough to install vulkan in the specifc deploy action scripts. Help and feedback is appreciated.

streamingdv · 2024-01-13T11:18:42Z

@saudet I'm not sure but I guess a ndk update is needed. Unfortunately it seems the build script depends on a lot of deprecated features

https://github.com/android/ndk/wiki/Changelog-r21

Some of them are no longer available in ndk 22 and above. I tried to play around with it in a different branch but no chance to get OpenSSL build with ndk 26. I already modified the Android patch as I think some of the stuff isn't needed anymore. Even though I update the path to include clang it still complains that clang can't be found.

If we can build ffmpeg with vulkan supported with ndk 21 it might not be needed but ndk 21 don't contain the vulkan headers so I think that's the reason why it always complain about vulkan requested but not found. Overall that seems to be not a trivial task and might need a lot of refactoring.

streamingdv · 2024-01-15T16:53:08Z

@saudet not 100% sure yet but I think vulkan decoding features are not fully supported yet by ndk 26 (latest release so far). The ffmpeg configure checks if the vulkan headers version is >= 255

https://github.com/FFmpeg/FFmpeg/blob/34a47b97de9d90a28ead36ca200f6a19a08ce2df/configure#L7216

The highest header version included by ndk 26 is 237. Which means to make it work a ndk upgrade is needed (currently ndk 21 is used) and even then I don't think these features will work as some headers and structures needed for the decoding are simply not yet included.

streamingdv · 2024-01-15T18:48:48Z

https://developer.android.com/games/optimize/vulkan-get-started#:~:text=Your%20game%20needs%20to%20include,time%20of%20the%20NDK%20release.

So I guess wait for ndk 27? I try some other things out but so far no luck. Including headers from the official latest vulkan SDK is clashing with ndk headers and is producing some weird errors during compilation. Is it possible to split this pull request up and include the Android vulkan support when ndk 27 is released at a later point? At least Mac, Windows and Linux can benefit from vulkan support in the meantime?

saudet · 2024-01-15T23:53:28Z

I suppose we can do that, but the builds are not working on Linux, Mac, and Windows, so please fix that

streamingdv · 2024-01-16T00:01:16Z

@saudet isn't that just because ffmpeg.yml workflow is using the the wrong action scripts for the steps
e.g.

bytedeco/javacpp-presets/.github/actions/deploy-ubuntu@actions

I updated the deploy action scripts for windows, ubuntu and macosx to install the vulkan sdk. As this is not done in your action scripts in the branch @actions the builds are failing of course.

Just tell me what is the correct approach of how to handle that and I will update my pull request.

streamingdv · 2024-01-16T00:16:37Z

@saudet my pull request adapted the deploy action scripts for ubuntu, windows and macos but I realize now that these changes most likely should be merged to the actions branch first (to include the installation of the vulkan sdk) and afterwards I can merge this branch with the master branch as the builds should no longer fail with the vulkan requested but not found error. Would this be the correct approach here?

saudet · 2024-01-16T07:32:01Z

Ok, I've temporarily added the changes to the actions branch and it looks like it's building fine, but we might need to bundle some dependencies... Could you show me what readelf -d libavcodec.so and dumpbin /dependents avcodec-60.dll look like? I guess we could temporarily add those commands to cppbuild.sh :)

streamingdv · 2024-01-16T08:28:12Z

Of course, I have tested it on my PCs. The dumpbin output is like that

Dump of file avcodec-60.dll

File Type: DLL

  Image has the following dependencies:

    ADVAPI32.dll
    KERNEL32.dll
    msvcrt.dll
    ole32.dll
    avutil-58.dll
    swresample-4.dll

  Summary

        1000 .CRT
      71C000 .bss
       17000 .data
        2000 .edata
        5000 .idata
       47000 .pdata
      633000 .rdata
       14000 .reloc
        1000 .rodata
        1000 .rsrc
     1B64000 .text
        1000 .tls
       66000 .xdata

The ouput of the readelf looks like this

Dynamic section at offset 0x212e908 contains 36 entries:
 Tag        Type                         Name/Value
0x0000000000000001 (NEEDED)             Shared library: [libswresample.so.4]
0x0000000000000001 (NEEDED)             Shared library: [libavutil.so.58]
0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
0x000000000000000e (SONAME)             Library soname: [libavcodec.so.60]
0x0000000000000010 (SYMBOLIC)           0x0
0x000000000000000c (INIT)               0xa7000
0x000000000000000d (FINI)               0x19d0934
0x0000000000000019 (INIT_ARRAY)         0x20a85d0
0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
0x000000000000001a (FINI_ARRAY)         0x20a85d8
0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
0x000000006ffffef5 (GNU_HASH)           0x260
0x0000000000000005 (STRTAB)             0x40e0
0x0000000000000006 (SYMTAB)             0x7b0
0x000000000000000a (STRSZ)              10684 (bytes)
0x000000000000000b (SYMENT)             24 (bytes)
0x0000000000000003 (PLTGOT)             0x2131000
0x0000000000000002 (PLTRELSZ)           10056 (bytes)
0x0000000000000014 (PLTREL)             RELA
0x0000000000000017 (JMPREL)             0xa4348
0x0000000000000007 (RELA)               0x7178
0x0000000000000008 (RELASZ)             643536 (bytes)
0x0000000000000009 (RELAENT)            24 (bytes)
0x000000006ffffffc (VERDEF)             0x6f60
0x000000006ffffffd (VERDEFNUM)          2
0x000000000000001e (FLAGS)              SYMBOLIC
0x000000006ffffffe (VERNEED)            0x6f98
0x000000006fffffff (VERNEEDNUM)         8
0x000000006ffffff0 (VERSYM)             0x6a9c
0x000000006ffffff9 (RELACOUNT)          26601
0x0000000000000000 (NULL)               0x0

You can also check yourself as I have built a test branch based on this branch which is uploading the .m2 folder for windows and linux after a successful build. This way you can just download ffmpeg-6.1-1.5.10-SNAPSHOT-linux-x86_64 and ffmpeg-6.1-1.5.10-SNAPSHOT-windows-x86_64.jar.

https://github.com/streamingdv/javacpp-presets/actions/runs/7535219075

.github/actions/deploy-ubuntu/action.yml

saudet · 2024-01-17T01:39:03Z

The build for Triton Inference Server is failing:
https://github.com/streamingdv/javacpp-presets/actions/runs/7549490770/job/20553566380
The script needs to be able to run under Docker with Jammy. If you've confirmed that the commented out lines work on Jammy, let's just uncomment that and use them when CODENAME == "jammy".

streamingdv · 2024-01-17T08:10:07Z

This is the new error?

https://github.com/streamingdv/javacpp-presets/actions/runs/7549490774/job/20553572039

Which uncommented line do you mean? Where exactly is the build failing now?

saudet · 2024-01-17T08:11:56Z

The lines you added to deploy-ubuntu/action.yml don't work on Jammy. That needs to be fixed

saudet · 2024-01-17T08:12:40Z

 Fetched 586 kB in 1s (791 kB/s)
Reading package lists...
W: https://packages.lunarg.com/vulkan/1.3.268/dists/focal/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 lunarg-via : Depends: libjsoncpp1 (>= 1.7.4) but it is not installable
 vulkan-profiles : Depends: libjsoncpp1 (>= 1.7.4) but it is not installable
 vulkancapsviewer : Depends: qt5-default but it is not installable
E: Unable to correct problems, you have held broken packages.
Error: Process completed with exit code 100.

streamingdv · 2024-01-17T09:03:01Z

Okay I will check, I have currently a test build running. I guess the fix will look like this

        # install vulkan sdk
        echo Installing Vulkan-SDK
        if [ "$CODENAME" = "jammy" ]; then
          curl -s https://packages.lunarg.com/lunarg-signing-key-pub.asc | $SUDO tee /etc/apt/trusted.gpg.d/lunarg.asc >/dev/null
          $SUDO curl -s -o /etc/apt/sources.list.d/lunarg-vulkan-1.3.268-jammy.list https://packages.lunarg.com/vulkan/1.3.268/lunarg-vulkan-1.3.268-jammy.list
        else
          curl -s https://packages.lunarg.com/lunarg-signing-key-pub.asc | $SUDO apt-key add -
          $SUDO curl -s -o /etc/apt/sources.list.d/lunarg-vulkan-1.3.268-focal.list https://packages.lunarg.com/vulkan/1.3.268/lunarg-vulkan-1.3.268-focal.list
        fi
        $SUDO apt-get update && $SUDO apt-get install -y libvulkan1 libvulkan-dev vulkan-sdk

streamingdv · 2024-01-17T09:09:48Z

Builds seem to pass with the changes mentioned above (ignore the incorrect branch name)

https://github.com/streamingdv/javacpp-presets/actions/runs/7553279625

https://github.com/streamingdv/javacpp-presets/actions/runs/7553088153

I will integrate these changes in the pull request and will push it. It has to be adapted in the actions branch as well. Will you do that?

saudet · 2024-01-17T12:13:34Z

Thanks! It's working: https://github.com/bytedeco/javacpp-presets/actions/runs/7373605597/job/20569555502

+) Enable ffmpeg vulkan support for Linux, Mac and Windows builds

9af3e0d

saudet added enhancement help wanted labels Jan 13, 2024

saudet marked this pull request as draft January 15, 2024 00:09

saudet reviewed Jan 16, 2024

View reviewed changes

.github/actions/deploy-ubuntu/action.yml Outdated Show resolved Hide resolved

saudet reviewed Jan 16, 2024

View reviewed changes

.github/actions/deploy-ubuntu/action.yml Outdated Show resolved Hide resolved

+) Reformat vulkan SDK installation steps in deploy-ubuntu action script

9bf8cd3

streamingdv changed the title ~~WIP: Enable vulkan for ffmpeg~~ Enable vulkan for ffmpeg (Windows, Linux, Mac) Jan 16, 2024

streamingdv marked this pull request as ready for review January 16, 2024 15:58

saudet added 2 commits January 17, 2024 09:43

Merge remote-tracking branch 'upstream/master'

432493f

Update CHANGELOG.md and fix nits

57e3be6

+) Add fix for installing vulkan sdk on jammy

4aef62a

Fix nit

2226fe9

saudet approved these changes Jan 18, 2024

View reviewed changes

saudet merged commit 99d110e into bytedeco:master Jan 18, 2024
17 checks passed

saudet removed the help wanted label Jan 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable vulkan for ffmpeg (Windows, Linux, Mac) #1460

Enable vulkan for ffmpeg (Windows, Linux, Mac) #1460

streamingdv commented Jan 12, 2024

streamingdv commented Jan 13, 2024

streamingdv commented Jan 15, 2024

streamingdv commented Jan 15, 2024

saudet commented Jan 15, 2024

streamingdv commented Jan 16, 2024

streamingdv commented Jan 16, 2024 •

edited

Loading

saudet commented Jan 16, 2024 •

edited

Loading

streamingdv commented Jan 16, 2024 •

edited

Loading

saudet commented Jan 17, 2024 •

edited

Loading

streamingdv commented Jan 17, 2024

saudet commented Jan 17, 2024

saudet commented Jan 17, 2024

streamingdv commented Jan 17, 2024

streamingdv commented Jan 17, 2024

saudet commented Jan 17, 2024

Enable vulkan for ffmpeg (Windows, Linux, Mac) #1460

Enable vulkan for ffmpeg (Windows, Linux, Mac) #1460

Conversation

streamingdv commented Jan 12, 2024

streamingdv commented Jan 13, 2024

streamingdv commented Jan 15, 2024

streamingdv commented Jan 15, 2024

saudet commented Jan 15, 2024

streamingdv commented Jan 16, 2024

streamingdv commented Jan 16, 2024 • edited Loading

saudet commented Jan 16, 2024 • edited Loading

streamingdv commented Jan 16, 2024 • edited Loading

saudet commented Jan 17, 2024 • edited Loading

streamingdv commented Jan 17, 2024

saudet commented Jan 17, 2024

saudet commented Jan 17, 2024

streamingdv commented Jan 17, 2024

streamingdv commented Jan 17, 2024

saudet commented Jan 17, 2024

streamingdv commented Jan 16, 2024 •

edited

Loading

saudet commented Jan 16, 2024 •

edited

Loading

streamingdv commented Jan 16, 2024 •

edited

Loading

saudet commented Jan 17, 2024 •

edited

Loading