No audio when playing two files after upgrading libmpv

[low-batt]

Important Information

Provide following Information:

• mpv version master
• macOS Version 13.6.3
• Source of the mpv binary or bundle my own build using mpv-build
• If known which version of mpv introduced the problem 0341a6f
• Possible screenshot or video of visual glitches N/A

Reproduction steps

• Using mpv-build run:
  • ./use-mpv-custom 0341a6f
  • ./update
  • ./rebuild
• Copy libmpv.2.dylib into iina/deps/lib
• Build IINA with updated libmpv
• Run IINA from Xcode
• Start a video playing
• Pause playback
• Open a second video in another window
• That video plays, but with no audio

As soon as playback of the first video is paused the following two messages are emitted over and over again in the Xcode console:

CompositeAudioConverter.cpp:1,236 Input data proc returned inconsistent 512 packets for 0 bytes; at 4 bytes per packet, that is actually 0 packets
CompositeAudioConverter.cpp:1,121 ProduceOutput: produced only 0 of 512 requested packets   Reinterleaver 0x6000011454a0
    Input:   2 ch,  48000 Hz, Float32, deinterleaved
    Output:  2 ch,  48000 Hz, Float32, interleaved
    Channel map: 0 1

With audio not playing if I change the audio device to a different set of speakers then the audio for the second video starts playing.

If I repeat the above steps, but switching to the commit before 0341a6f:
./use-mpv-custom 1888591

Audio works and the above errors are not seen in the console.

An IINA user tried upgrading libmpv, encountered this problem and reported it in iina/iina#4789

Expected behavior

Audio is heard through the speakers.

Actual behavior

No audio is heard. Audio related errors are reported in the Xcode console.

Log file

mpv.log

Sample files

Any two videos with audio tracks should reproduce the problem.

[low-batt]

@handlerug Thoughts on this regression?

[handlerug]

Right, I never checked the console — I see those messages too. To be honest, I don't know what to do. I have no experience with the Core Audio API at all. Seems like Core Audio really wants the exact number of frames, and I don't see any other way to override that number. I guess the AudioBuffer hack doesn't actually work.

This should fix it (checkout mpv master first), but that means we will block in the render callback:

diff --git a/audio/out/ao_coreaudio.c b/audio/out/ao_coreaudio.c
index 484c31b31d..988dfeb039 100644
--- a/audio/out/ao_coreaudio.c
+++ b/audio/out/ao_coreaudio.c
@@ -87,13 +87,7 @@ static OSStatus render_cb_lpcm(void *ctx, AudioUnitRenderActionFlags *aflags,
 
     int64_t end = mp_time_ns();
     end += p->hw_latency_ns + ca_get_latency(ts) + ca_frames_to_ns(ao, frames);
-    int samples = ao_read_data_nonblocking(ao, planes, frames, end);
-
-    if (samples == 0)
-        *aflags |= kAudioUnitRenderAction_OutputIsSilence;
-
-    for (int n = 0; n < buffer_list->mNumberBuffers; n++)
-        buffer_list->mBuffers[n].mDataByteSize = samples * ao->sstride;
+    ao_read_data(ao, planes, frames, end);
 
     return noErr;
 }

On the other hand, I see a few uses of ao_read_data() when running git grep '\Wao_read_data\W', so perhaps it isn't so bad.

[low-batt]

Thank you very much for responding!

I pulled the latest master (fe0c181), applied the above patch, built libmpv and tested it using IINA. Audio worked fine. The messages seen in the Xcode console are no longer being emitted.

I read the discussion in PR #12643 about audio crackling on macOS. I have experienced crackling. I started paying attention to it and found I was only able to reproduce it when running under Xcode. It happens when I first open the IINA menu. This was before all of the latest changes. With the latest changes it still crackles, with or without the patch.

Since we are discussing audio on the Mac, another important problem to be aware of is issue #11617.

In testing with master I have encountered some sort of severe regression with playing an AV1 encoded video. I didn't find any open issues. Maybe my own build issue. I will investigate.

[krackers]

I think the situation is more subtle than CoreAudio unilaterally not allowing you to override the requested number of frames. If you look at Apple's own reference example in https://developer.apple.com/library/archive/samplecode/CAPlayThrough/Listings/PublicUtility_CARingBuffer_cpp.html#//apple_ref/doc/uid/DTS10004443-PublicUtility_CARingBuffer_cpp-DontLinkElementID_16 you can see that they implement a ring buffer similar to mpv and overwrite mDataByteSize based on number of samples that were actually returned.

And many "canonical" examples of using core audio render output callback overwrite mDataByteSize. For instance https://github.com/Epskampie/ios-coreaudio-example/blob/master/IosAudioController.m#L76-L107 which comes from https://atastypixel.com/using-remoteio-audio-unit/

However I also found https://github.com/Provenance-Emu/Provenance/blob/4cd4d3f7785b2498668734ea7825f5b80d180b26/Cores/PPSSPP/PVPPSSPPCore/Core/iOSCoreAudio.mm#L51-L63

which is maybe where the trouble starts. It could well be possible then when apple started rewriting audio subsystems (possibly copying things over from iOS) they began making some assumptions about the number of returned samples. (In fact I disassembled audiotoolbox framework from older version, and that warning message is nowhere to be found).

Now in my case, I can see those log messages even with just 1 mpv process running, it happens when I seek around (since at that point buffers are not completely filled yet). However unlike IINA when I launch 2 separate mpv processes they can both play audio in parallel fine (and I don't see repeated error messages).

Given that in IINA both audio units are on separate threads (compared to 2 separate mpv instances which puts them in separate processes), perhaps on newer macOS there are some strengthened assumptions that if you have multiple audio output units in a process, they must not overwrite number of samples? One could possibly create a small test program to recreate this behavior (launch two threads each playing a different sine wave), and try to see if it's a particular macOS version which broke this.

[low-batt]

I pulled the latest master, built libmpv and tested playing multiple files with IINA. All working fine.

Many thanks to everyone who worked on this issue.