Support for Spatial Audio on macOS Monterey

[leolabs]

What you want IINA to do:

It would be awesome if IINA could passthrough surround / Dolby Atmos streams to connected AirPods so they can make use of the new spatial audio features in macOS Monterey.

What IINA does currently:

Currently, IINA downsamples surround streams to stereo when connected to AirPods.

Why you think this should be added:

It would enhance the movie watching experience.

Examples of other projects that have something similar:

Since this is a fairly new feature, I've only seen QuickTime, Safari, and their TV app support this so far.

[tbodt]

mpv-player/mpv#9252

[iGerman00]

Any news on this? I cannot find any app whatsoever that's not an Apple or Catalyst app that can even do spatial audio. Not even Safari can do it with Plex or Netflix, at best its spatialize stereo but not even that is available on most apps...

[noelboss]

This would be awesome!

[richtong]

Yes, please do, with the latest HDR branch #3539 which does HDR well, this is the last missing piece. Please :-) The list is small but growing https://www.macrumors.com/guide/apps-support-apples-spatial-audio-feature. And there are no MacOS apps yet (Air Video HD is iDevice only :-)

• Air Video HD (Turn on Surround in Audio settings)
• Apple's TV app
• Netflix
• Disney+
• FE File Explorer (DTS 5.1 unsupported)
• Foxtel Go (Australia)
• HBO Max
• Hulu
• Plex (Enable old video player in Settings)
• Peacock
• Discovery+
• Paramount+
• Vudu
• Some TIDAL songs

[mjbaldwin]

Just wanted to +1 this -- I had to stop using IINA and started using Infuse because it's the only Mac player that will output spatial audio on my M1 Mac with AirPods on my MKV files.

Would love it if IINA supported this.

[mjbaldwin]

It is just AC3/EAC3, since that's the only stream type Apple spatial audio supports.

It's not TrueHD or AC4. Those would require audio transcoding to EAC3 which doesn't seem to be a reasonable expectation for a video player right now. (If you really want spatial audio for those, you could use ffmpeg to transcode file audio to EAC3.)

[madhavtummala]

Infuse won me over, it supports spatial audio (even atmos output), can connect to my plex/jellyfin server and can also do trakt sync! And its the only M1 native app that can do this!

[dasti555]

Yes, please do, with the latest HDR branch #3539 which does HDR well, this is the last missing piece. Please :-) The list is small but growing https://www.macrumors.com/guide/apps-support-apples-spatial-audio-feature. And there are no MacOS apps yet (Air Video HD is iDevice only :-)

• Air Video HD (Turn on Surround in Audio settings)
• Apple's TV app
• Netflix
• Disney+
• FE File Explorer (DTS 5.1 unsupported)
• Foxtel Go (Australia)
• HBO Max
• Hulu
• Plex (Enable old video player in Settings)
• Peacock
• Discovery+
• Paramount+
• Vudu
• Some TIDAL songs

Optimus Player does this.

[byGits]

Yes, please do, with the latest HDR branch #3539 which does HDR well, this is the last missing piece. Please :-) The list is small but growing https://www.macrumors.com/guide/apps-support-apples-spatial-audio-feature. And there are no MacOS apps yet (Air Video HD is iDevice only :-)

Infuse Pro on App Store supports both Spatial Audio (head-tracked and fixed) and HDR.

But it is a paid app, $1/m and TBH not as good as IINA.

[dasti555]

Look comment above. And it's a one-time purchase only.

[mjbaldwin]

Just wanted to check if there's been any progress towards this.

In the MPV issue it looks like another player, KSPlayer, has possibly added the relevant code to support spatial audio? See mpv-player/mpv#9252 (comment)

[brownknightrises23]

Any update on this?

[brownknightrises23]

mpv-player/mpv#11955

[Crear12]

Please, Spatial Audio + Vision Pro playing videos with 6/8 channel audio would be next decade experience!

[godly-devotion]

Since AVKit (and QuickTime Player) supports Spatial Audio I decided to quickly patch together a super basic video player that supports HDR and Spatial Audio playback for multichannel audio as an interim solution for me.

https://github.com/godly-devotion/FrontRow

[kingslay]

Can use the TracyPlayer APP. It supports HDR and Spatial Audio.
https://apps.apple.com/us/app/tracyplayer/id6450770064

[koochey]

+1

[JulesLePrince]

This commit adds AVFoundation support to mpv. I tested it and spatial audio is working well.

[ShlomoCode]

mpv now supports spatial audio!
mpv-player/mpv#9252 (comment)

[Rozzemak]

So,.. if I update mpv via brew to get the latest brew binaries and symlink them to IINA, it should be automagically working ?

Basically very similar to this:
(https://www.svp-team.com/wiki/SVP:IINA)

[ShlomoCode]

It is now included in official mpv version - https://github.com/mpv-player/mpv/releases/tag/v0.38.0
@low-batt do you know when there will be an iina version that includes the latest mpv version?

[low-batt]

My first reaction is that I am disturbed at the release of 0.38.0 as the mpv issue I entered about audio on the Mac not working for apps that use libmpv and open more than one file (IINA issue #4789) is still open. If that is not fixed then IINA will have to patch mpv which I was hoping IINA would not have to do. I will have to test the release.

Glad all of you have been following the mpv progress and brought this to my attention. I've been too busy with IINA code lately to keep an eye on mpv changes.

I am a junior IINA developer so I can only speak to what I think the release plans are…

The 1.3.4 release was rushed in order to meet an Open Subtitles end of 2023 deadline. Quite a few fixes missed the boat. So the plan is to follow up with a 1.3.5 release. This is intended to be a stabilization release and therefore unlikely to contain major changes. If doable this release will restore support for macOS 10.11 and 10.12. For that reason IINA will not be upgrading libmpv as doing so would require IINA to drop support for macOS 10.11-10.14 as mpv has raised the minimum supported macOS to 10.15. There are a few PRs that still need to be reviewed and merged for 1.3.5. The hard work left is to get IINA working under macOS 10.11. I don't know the status of that work.

After that the plan is to start on the long delayed feature release. This release will include betas. The new plugin system will be enabled for this release. This will also include some fixes that are more risky to make (for example threading problems) that have been postponed for a release that includes betas. There is also a bunch of PRs ready for review that add small features. My own preference would be to put as much as possible into the first beta rather than dribble out features over a series of beta releases. This release will definitely include upgrading mpv and FFmpeg. So IINA will have to raise the minimum supported macOS version to 10.15 for this release.

As for when these releases will come out, I don't have a guess on that. Other than the 10.11 issue I don't see a lot of work to get 1.3.5 out besides the normal work needed to produce a release. I see a lot of work to get the outstanding PRs merged for the first beta. Upgrading dependencies is always "interesting". When that comes out depends upon decisions about what will be included in the feature release. That hasn't been nailed down yet. Need to first get pass 1.3.5.

[ruihe774]

My first reaction is that I am disturbed at the release of 0.38.0 as the mpv issue I entered about audio on the Mac not working for apps that use libmpv and open more than one file (IINA issue #4789) is still open. If that is not fixed then IINA will have to patch mpv which I was hoping IINA would not have to do. I will have to test the release.

We are discussing a fix for it now (mpv-player/mpv#13902 (comment)). I think we are going to fix it soon. Sad that the fix did not catch up with v0.38 release. You may do some backporting afterwards.

ao_avfoundation is a new ao that supports spatial audio. You may want to test it, and IMO you can make it the new default for IINA giving the lack of a maintainer of old ao_coreaudio.

[low-batt]

I confirmed mpv-player/mpv#13348 still reproduces with the latest mpv master.

I did a very quick test with ao_avfoundation and did not encounter a problem.

[low-batt]

@ruihe774 Thank you very much for the link to the Apple examples. I had searched for Apple examples but ended up getting lost in Apple Music.

@ShlomoCode Did you strip the .txt off the end of the two configuration files I posted and use them in your mpv-build build?

The library should end up in a mpv/build subdirectory as seen here:

low-batt@gag mpv-build (master %=)$ ls -la mpv/build/libmpv.2.dylib
-rwxr-xr-x  1 low-batt  staff  30528048 Apr 19 19:46 mpv/build/libmpv.2.dylib
low-batt@gag mpv-build (master %=)$ 

The mpv_options file sets libmpv to true:

-Dbuildtype=debug
-Dlibmpv=true
-Db_asneeded=false
-Dc_link_args=-fuse-ld=/usr/bin/ld
-Dcplayer=false

Without that the mpv app will be built instead.

[ShlomoCode]

mpv/build/libmpv.2.dylib

Thanks! That's what I was missing. I just couldn't locate the file...
now I copied and replaced the files from mpv-build/mpv/build to iina/deps/lib and built, and I get Spatial Audio with iina! Wow this is wonderful 💫

[ShlomoCode]

Update: I changed in Build Settings > Architectures to "arm64" only and it solved the error for me.
@low-batt Running Xcode development (▶️ button) works and I can run Spatial Audio, but when I try to generate an .app file to put in my Applications (Product > Archive in Xcode), I get an "Undefined symbols" build error:

[low-batt]

Glad to hear your build worked! And yes mpv-build does not generate a universal library like the libmpv provided by download_libs.sh. The mpv-build generated library is only usable for the architecture it was built under.

[low-batt]

To get spatial audio to work you set ao on the Advanced tab of IINA's settings as I mentioned above, yes?

[ShlomoCode]

Yes, Or the change presented at #4909. Both worked for me (with Airpods Pro 2).

[dasti555]

Heh, so when OTA for non-dev people? 😁

[low-batt]

On this:

Heh, so when OTA for non-dev people?

See this comment above on what I think the release plans are. I'm a junior IINA developer and can't comment on when the releases might happen. I can only say that work is ongoing.

[Sponge-bink]

I was able to make IINA output Spacial Audio by temporarily replacing mpv that came with IINA with a version from Homebrew. Here are the steps for someone who needs them.

Backup the original mpv that came with IINA:

cp /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib /Applications/IINA.app/Contents/Frameworks/libmpv_back.2.dylib

Then delete the original mpv that came with IINA:

rm -f /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

Install mpv if you haven't:

brew update && brew install mpv

Create a symbolic link to the Homebrew mpv (0.38.0 as of now) where the original mpv was:

ln -s /opt/homebrew/Cellar/mpv/0.38.0/lib/libmpv.2.dylib /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

Then perform the action demonstrated by @low-batt. Now IINA should be capable of playing in Spacial Audio.

And if you want to undo it:

Remove the symbolic link you created:

rm -f /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

Recover from the backup:

cp /Applications/IINA.app/Contents/Frameworks/libmpv_back.2.dylib /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

You can leave the backup there or delete it:

rm -f /Applications/IINA.app/Contents/Frameworks/libmpv_back.2.dylib

At last, don't forget to undo the process of setting the ao option of mpv to avfoundation. Otherwise there will be no audio.

[ShlomoCode]

@low-batt I found a bug (seemingly) that only occurs when ao is set to avfoundation: when you drag a file into iina, it doesn't start playing by itself. The playback position must be changed and only then the video starts playing.
A detailed demo video is attached with:

1. The latest iina code from develop (73c9608)
2. Updated mpv dylb from mpn-build

CleanShot.2024-05-08.at.01.45.49-converted.mp4

[ShlomoCode]

In addition, when Spatialize Stereo is turned on/off, there is a skip of about a second in the audio. With QuickTime it doesn't happen:

Screen.Recording.2024-05-08.at.2.10.39-converted.2.mov

[low-batt]

I've created issues #4920 and #4921 for the problems. Let's continue the discussion in those issues.

[gingerbeardman]

Thanks @Sponge-bink

[ShlomoCode]

@uiryuu Note that #4920 is not yet resolved.

[uiryuu]

This issue is closed automatically by GitHub. Merging #4909 is for testing purposes, which doesn't mean we have solved every problem. We'll keep an eye on #4920.

[Willian-Zhang]

For anyone who is still watching this thread, use @Sponge-bink's method replacing mpv, I have made this replaced version of IINA (use at your own risk) (on macOS 14.6.1, arm)

link removed

check the conversation below

if it says this file is broken, run this and right click open:

xattr -c /Applications/IINA.app

and @low-batt's method enabling it:

check the conversation below

after setting it, quit and restart the iina.app to see the changes

[uiryuu]

@Willian-Zhang You can also visit nightly.iina.io to download the latest nightly build, which contains a button to choose from avfoundation and coreaudio.

[Willian-Zhang]

@Willian-Zhang You can also visit nightly.iina.io to download the latest nightly build, which contains a button to choose from avfoundation and coreaudio.

wow thank you for the info, this is way better

[Willian-Zhang]

For anyone who is trying to find the option, its here:

[Sponge-bink]

I have made this replaced version

@Willian-Zhang I post that method so no one will have to trust other people’s build to use this feature…

[Willian-Zhang]

I have made this replaced version

@Willian-Zhang I post that method so no one will have to trust other people’s build to use this feature…

Thank you for the heads up.
and yes I intentionally posted the modified binary for those who wants this feature yet not comfortable dealing with homebrew.
now I removed the link, since there is an official nightly build.