Device Notification - Event or callback for arrival/removal of devices

[msiglreith]

WASAPI or PulseAudio provide an event subscription mechanism for detecting for example arrival or removal of devices.

[Be-ing]

If CPAL implements support for this, that alone would justify the cost of figuring out how to use CPAL from Mixxx which is written in C++. As far as I can tell, crossplatform audio hotplug is an unsolved problem. There were efforts to do this in PortAudio years ago but they were not finished or merged. RtAudio does not support this either.

This is crucial for live performance software. Having to restart the application when someone trips over a USB cable is very bad! (It's also a shame that we are stuck with physical connectors that are intentionally designed to be easy to unplug, which is the opposite of what we need on stage...)

[Be-ing]

There are three pieces to this puzzle:

1. Detecting device disconnection. This is already implemented for most backends with the DeviceNotAvailable error.
2. Detecting device connection.
3. Associating a newly connected device with a device that was unplugged before. This may be the hardest part.

[hrydgard]

The most basic functionality here, but still very desirable, would be an option to automatically "follow" the OS default device when it changes. This could maybe be hidden internally so that a more generic notification mechanism doesn't need to be created, initially at least - although I guess when the new device have different sample rate or format restrictions it could get dicey... Hm.

[Ralith]

+1, I'd love to be able to respond to changes in the default device. It would be especially helpful if cpal provided a way to move the callbacks between devices, so I don't have to rebuild all my inter-thread communications channels from scratch each time.

[generalelectrix]

@Be-ing with regards to "Detecting device disconnection. This is already implemented for most backends with the DeviceNotAvailable error." - unless I'm mistaken, this doesn't seem to be implemented for CoreAudio? The only place a stream's error_callback is invoked is this line:

cpal/src/host/coreaudio/macos/mod.rs

Line 673 in 9748543

error_callback(err.into());

But StreamError::DeviceNotAvailable does not appear to actually be constructed anywhere.

Probably related to my question/bug #704

Looks like rtaudio implements this feature in this section of code:
https://github.com/thestk/rtaudio/blob/master/RtAudio.cpp#L1907-L1916

[generalelectrix]

Overall this feature seems at least reasonable to implement. Device reconnection is not very hard to solve - you're already dedicating an entire thread to audio processing, and if the device disconnects, that thread is no longer busy. Even without a "device added" hook on the backend, once the device disconnects you can just poll the create action with the same name periodically until the device reappears. The main trick here is getting ownership of the processing and error closures back from the processing thread. This implementation would need to be backend-specific, since the handling of the closure is different in every case.

Support for this definitely exists in Core Audio, as the AudioUnit implementation already has methods that free the processing callback and return ownership.

WASAPI looks straightforward as the audio handling thread is created in this library, so handing the closures back over a channel or something when the processing event loop terminates is straightforward. Another option would be to internally wrap them in a mutex so there's no need to explicitly pass ownership back from the processing thread, and re-use them to create a new stream. Since in practice they won't be accessed by more than one thread at a time, the mutex should remain uncontested and imply negligible overhead.

ALSA looks similar to WASAPI. ASIO passes ownership of the callback into a thread owned by generated library code, and I can't see if there is a similar mechanism for returning ownership of the callback.

emscripten... it's weird in here. I'm guessing that a mutex is not meaningful in this context, and I'm unfamiliar with the concurrency primitives available that might be able to return ownership of the callback. That said, it is currently passed into the audio processing thread via a raw pointer; I guess ultimately it leaks? Though I'm a bit surprised that it isn't dropped when this function returns.
https://github.com/RustAudio/cpal/blob/master/src/host/emscripten/mod.rs#L213-L232

JACK takes ownership of the callbacks and there's no existing mechanism to return them.

After rummaging through all this code, and also considering that the processing callback may have assumptions burned into it about device parameters that could change during reconnect, if I were tasked with solving this problem, I would not try to implement it at the level of this library. It seems like there is too much inherent complexity in how the callbacks are managed, and the assumptions present in those callbacks may no longer be valid after device reconnection.

[generalelectrix]

My suggestion to future people reading this issue - consider implementing reconnection in your application, because any implementation provided at the level of CPAL will probably need to make assumptions that your application may not be OK with, and you may also want to provide semantics beyond straightforward "reconnect to the same device" such as explicit fallback.

Basic reconnection is not difficult to implement at the application level. Here's an example I extracted from a project that uses a monitor thread to handle automatic reconnection to the same device, for inputs. Creation of the callback function is delegated to a factory closure. This could be tweaked to accept the device and/or config to ensure the resulting handler closure is configured correctly for what may be a new set of device settings. In my actual project I just dumped the handler creation code directly into create_input_stream rather than passing in a closure.

https://gist.github.com/generalelectrix/0933c19e7efd82af330d8c940e03ec9b

Note that this implementation won't return an error if the initial device open fails. This was because Stream isn't Send, so it must be created in the monitor thread. This could be fixed by performing the initial open in the monitor thread, sending back a Result<(), BuildStreamError> to the original thread via a one-shot channel, then blocking the original thread until receipt of that message to provide the return status.

Also note that this mechanism relies on the backend terminating the stream and sending your error callback StreamError::DeviceNotAvailable which not all backends may do. Seems like the major missing piece preventing at least cross-platform "device is gone" notifications is ensuring that every backend provides the guarantee of this behavior. #707 adds this for CoreAudio on MacOS.