Server pipe out the audio to some DAW software for mixing #146
Comments
|
I stream audio to and from Reaper to Jamulus now. It's not something that needs adding to Jamulus. Many people already use Jamulus with Jack on Linux - and you should be able to use JackRouter on Windows. http://jamulus.drealm.info/jamulus Note that the client receives stereo, so there's no opportunity to remix outside Jamulus. If you want the audio being uploaded, you need to run a server that informs the people connecting that you're recording their property. The server includes the ability to record. |
|
Thanks @pljones, just to clarify; are you saying that it is already possible in the Jamulus server to pipe all channels of the clients audio to a DAW like Reaper? It sounds like you're describing how a client could send audio to a DAW before sending it through Jamulus on to the server. I'm talking about the server itself using a DAW with all channels of the clients audio. Does that make sense? Or even sound like a good idea? |
|
The server has no sound I/O itself. It can write files containing the audio that can be read by a DAW (indeed, it writes a Reaper RPP to go along, which lines all the files up in the time line correctly - the file names also encode the position). Remember, time is critical for Jamulus. The problem with passing the data from one client to each other client, unmixed is that each time a new client joins, the volume of data uploaded by the server increases exponentially, so the delay in sending to each client will similarly increase. Currently, the increase is linear - as it's always only a stereo pair being added. That exponentially increase will exceed the available time (i.e. the duration of a packet) very rapidly, meaning packets start getting dropped constantly to keep up. Emitting audio data in real time (i.e. uncompressed, taking one packet's time line time per packet) would be even worse - each packet received would immediately take a packet length of time to play back, meaning no time left to do anything else. |
|
pljones: It's true, but mixing the incoming client audio also takes time. It would be nice to have the per-client option for the server to send the streams synchronized but unmixed back to that client. In my situation (fast symmetrical Internet connection, and running my client on the same computer as the server) it wouldn't be a performance problem. I understand the exponential increase would be a problem for some but I'd like at least the option of a client getting all the client streams back unmixed. Then whoever is producing the livestream of the performance could mix it down in their DAW. Another way to do it might be to have the option of the server doing its mixing by feeding all the audio streams to an external-but-local DAW for realtime processing and mixing, and then back into the server to distribute to all the clients. I hate to see Jamulus re-inventing all the mixing stuff that a DAW does just because Jamulus won't co-operate with other implementations of that functionality. |
|
The Jamulus server needs to mix every client down to stereo for network performance. If you do not have network performance issues, then Jamulus may not be what you're looking for. There is software, like the network Jack connectors and network ASIO devices, that let you connect computers directly together over a network and into a DAW where there is no significant network delay to worry about. |
|
@pljones Well, network performance is not an all-or-nothing thing. My Internet connection has limits, but they are big enough that I could handle some reasonable use cases while implementing the feature suggested. For example, let's assume 200kbps per stereo stream. IF everyone has to have unmixed copies of all the streams, then the server on my 60Mbps up/60Mbps down connection could handle up to 16 clients. That's more than I've ever seen on any public server, and seeing as I don't know 16 people who I want to use Jamulus with for my specific application, that's not a problem for me. But I'm not proposing that EVERYONE has to have unmixed copies of all the streams, and we sure don't want to limit Jamulus to just 16 clients. I just want a per-client option as to whether the client gets back the mix or the not-mix. That would take quite a bit less bandwidth, and if the client is running on the same computer as the server, it would be almost free of performance cost. In addition, because Opus does multichannel encoding, it's likely to be less bandwidth than num_clients*200kbps for that client. Practically speaking, to recap, what would help me is either: Both options would effectively allow removing the recording code from the server, which would be a small win. (a) would require everyone to hear the same mix with effects, EQ, etc. while (b) would not. I'm aware of the other software such as jack_netsource and VSTConnect, but they are not something I can send to a singer acquaintance and just say "install and run this on your Mac". Jamulus is fairly approachable for non-technical users, and that's a good thing. :-) You're right to keep Jamulus performance in mind, because its ability to do what it promises depends on that. It's just a matter of finding a way to keep the performance and simplicity while allowing for some of these more advanced use cases. |
|
Acting as an audio service client is non-trivial... It would need an asynchronous component receiving the pushed raw audio, similar to the audio recorder. That would, at its leisure - to avoid impacting main Jamulus performance - push audio out to whatever audio interface it's connected to. Given most audio services don't like "asynchronous" and "at its leisure", the component would be decoupling the main Jamulus server from the audio service. Due to the lack of synchronisation, that would mean dropping audio service buffers, most likely. It would also need to be cross platform and work from the GUI and command line. Obviously it would need to be optional to enable at all on the Jamulus server side - most server operators would likely not want it, similar to jam recording. Again, similar to jam recording, I would recommend instating the server licence notice if this was in use, as Jamulus client user's audio would be being pushed outside Jamulus -- doing so without their knowledge would be "not nice". |
|
Sounds like you're talking about option (a). What are your thoughts on
option (b)?
|
|
(b) Non-viable. |
corrados
changed the title
|
Is there a way to push the stereo output to icecast on the server side without using another windows/mac client to route? |
If I'm not mistaken, the only way to get audio out directly from the server at this time is the recording interface. I've never tried to use it as a route or pipe, but maybe someone else did. [Possible enhancement] I've never had time to try it out, but I think using a Reastream (https://www.reaper.fm/reaplugs/) out interface would be the easiest way of getting real-time audio in some other tools. Reastream being a free VST v2 can easily be used as input in any DAW around. The decompressed audio is already available inside Jamulus, so it will be a matter of packing it correctly in UDP and transmit to the receiving pgm. The protocol seems simple enough (UDP based, simple format, supports up to 64 channels): |
|
The easy way is to sit a client in the server, use its mixer (possible headless with MIDI control) and have it stream, much like WorldJam does and much like the various NINJAM servers do. The server isn't intended to do this kind of thing. The server recordings can be mixed after the event with one less network hop, of course. |
|
Another option is to sit a second client in the server for each client channel you want to export for mixing, route that client into a DAW and do your mixing there. That way you get the individual clients separated for treatment (e.g. high pass filter to reduce the bass muddiness on the drums, or whatever). Of course, you need the network bandwidth for that. |
|
Based on the information I compiled before (#146 (comment)) on Reastream, I did a quick capture between Reaper (receiver) and Audacity (sender) of a 48K wav file to validate the protocol information and it matches OK. A 48KHz stereo stream produces a stream of UDP packets, at a rate of 1 packet every 0.2 msec (which seems a bit high and may need an additional check). A quick and dirty test on python validated that the Reastream VST on Reaper connects to any stream using that protocol, no loss of audio quality noticeable. No handshake occurs, just packets of data. I believe this feature will be a wonderful addition to Jamulus, as it will provide a no audio loss and lower latency (due to the intermediary client as used today, implying an extra mix and encode/decode) stream to a DAW or direct to OBS (or any other audio tool capable of using VSTs). As an implementation idea, I imagine it can piggyback on existing recording code (where the uncompressed audio buffers exist already) and if there's no need to send all separated channels (or the performance penalty is an issue), a director stereo mix (as recently discussed in #599) can be also used as a mix down stream for streaming. Thoughts? |
|
Given the issue title
I don't think the #599 director mix is relevant -- quite the opposite. Also, if the protocol is limited to 64 (mono?) channels and Jamulus server already supports around that number (in stereo) and is looking to increase capacity, wouldn't this cause another conflict? I don't think the recorder is the right place to put it. That's for recording. This isn't recording. The AudioFrame signal that the recorder listens for can be used by other slots, as well. So it would be better to do anything related to this in a separate handler. |
Each Reastream "session" is limited to 64 channels per connection. But if you change the identifier, you can have another stream of 64 channels and so on... the networking stack will surely scream :-) but still technically possible.
Agreed, I assume Recording isn't the right place, but just mentioning it because of code reusability and data readiness (they both need the same uncompressed audio data, one for storing it to a file and the other for streaming). |
|
FWIW, I have run a Jamulus client on a second computer, then piped the output to a Virtual Audio Driver and on to Microsoft Teams, Zoom, etc. Instructions are in this doc... https://docs.google.com/document/d/1smcvsxdaaViPQvGMQHmah_6BQeqowhmGSFMHfnlY2FI/ |
|
Hi all - I think this can be closed now that the new streamer is in place. If you want discuss further feel free to do so in Discussions. |
Firstly, this software is great. We've been using it to practice and record performances together.
I have a request for a feature that isn't quite in Jamulus but may not need too much to make it happen, but I really don't know. And I want to see if this is something that others are interested in too (while I think this feature would be cool others might have better ideas).
I had the thought that it would be good if there was a way to have the server pipe out the audio it receives from clients to some DAW software for mixing either for a live stream, and/or in order to send it back to clients processed.
Would it take much to add this in? Initially it could be for Jack on Linux. It may be similar to how the server pipes out audio for recording.
Again really impressed with the software and development. Looking forward to what plans your have for it.
The text was updated successfully, but these errors were encountered: