-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Record Desktop Audio #1079
Record Desktop Audio #1079
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
Does this mean the resulting media container will have two audio streams? Or is it one stream with a mix of both audio sources? Depending on how exactly this works, I think this could cause serious problems. |
The resulting file contains one audio stream with a mix of both audio sources. |
Okay, that's definitely better than the alternative having two audio streams in one container. Still, while this might be a bit more “professional”, I think it may cause some trouble, and it may be easier to just have a mix of all audio streams in all video containers. The current patch failing to play back audio correctly right now is a good example. For further processing, you now also need to make sure in some cases to mix audio afterward. It would be so much easier to just have the same audio everywhere. |
opencast-studio/src/steps/review/preview.tsx Lines 128 to 132 in 3621089
So: you can just remove that (or make it conditional). Currently, CI fails and there is no test deployment. Can you fix the issues and push again? There are also style lints that are visible here on GitHub.
I'm unsure about that. I do have remind myself of the target audience of Studio, yes, you are right. I have one specific situation where separate audio streams are useful: if a user recorded a long session and only notices in the end that the microphone is way louder (or quieter) than the desktop audio. To make a useful video out of that, one would have to boost on of these tracks. With both tracks already combined that's basically impossible. Yes, this boosting would have to happen outside of Studio but at least it's possible. One idea that was floating around in our chat: add both. The first audio track of both files is the "all sources combined" audio. And then we still add a second audio track that only has desktop audio or microphone. Players default to playing the first audio stream, so normal users wouldn't notice it. Sure, the file size would be slightly larger and the recording would require slightly more resources. Dunno, I guess I don't have a strong opinion 🤷 |
This comment was marked as resolved.
This comment was marked as resolved.
This PR allows Studio to record desktop audio and resolves elan-ev#551 Unfortunately, not all browsers are supported yet, as you can see here: https://caniuse.com/mdn-api_mediadevices_getdisplaymedia_audio_capture_support and as Lukas mentioned here: [elan-ev#551](elan-ev#551) The users get a hint after selecting display as source, with which browsers they can record desktop audio. Output of the recordings for download: Only camera recorded: Nothing changed, the finished camera video will contain microphone audio (if recorded) Only display recorded: If the user recorded both, desktop audio and microphone audio, the finished video will have both audio tracks. Display and camera recorded: The finished desktop video will contain desktop audio and the finished camera video will contain the microphone audio One thing/problem: In the "Review & trim" step (also after uploading to Opencast), if both the display AND the camera were recorded: you only hear the microphone audio or nothing (depending on whether microphone audio was recorded or not). I'm not entirely sure why. The audio is correct and audible for each of the videos that can be downloaded in the last step in Studio.
e825987
to
db735d1
Compare
Rebased and fixed conflicts. |
There is a test deployment now, but the code is subject to change, still. FWIW: https://test.studio.opencast.org/2023-11-16_12-41-36-pr1079-6890854908/
For the record: This I can't confirm. Is this a platform issue? Can others reproduce this? EDIT: Ah, nevermind. The all-caps "AND" was important. Who would've thunk. 🙈 |
OK that bug should be fixed now, and I had to change a few other things, but this should work: All the audio tracks are now mixed and added as single audio track to all the resulting videos. |
With this, Studio pre-mixes all the audio tracks from all the recordings, i.e. desktop and mic, pre-mixes them and uses the resulting single audio track in the videos that are uploaded to Opencast, and those the user can download. This also works in the review step.
8f4056e
to
c1322d8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only small stuff. Otherwise this looks good. Will test the deployment now.
c1322d8
to
83df85b
Compare
83df85b
to
a6faf88
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything looks good! I think I now tested every possible combination of video-source and microphone and seems like the output files are exactly as expected.
Namely, as we recently decided: all available audio sources (display audio, microphone) are combined together into one track and every output video file gets that exact audio track. That will likely be the option leading to the least user confusion. All videos file will be useful. Only small disadvantage is that it's not possible to separate the two sources again, e.g. to make only the microphone input louder. We can still allow users to download the individual audio tracks in the future.
We both noticed that the audio indicator UI can be fairly hidden, depending on what was shared. But that's the same for the settings button, so we will fix that later.
This PR allows Studio to record desktop audio and resolves #551
Unfortunately, not all browsers are supported yet, as you can see here and as Lukas mentioned here: #551
The users get a hint after selecting display as source, with which browsers they can record desktop audio.
Output of the recordings for download:
Only camera recorded: Nothing changed, the finished camera video will contain microphone audio (if recorded)
Only display recorded: If the user recorded both, desktop audio and microphone audio, the finished video will have both audio tracks.
Display and camera recorded: The finished desktop video will contain desktop audio and the finished camera video will contain the microphone audio
One thing/problem:
In the "Review & trim" step (also after uploading to Opencast), if both the display AND the camera were recorded: you only hear the microphone audio or nothing (depending on whether microphone audio was recorded or not). I'm not entirely sure why.
The audio is correct and audible for each of the videos that can be downloaded in the last step in Studio.