Feature request: Add audio volume control and sound split commands

[mltony]

I propose to add the following commands related to audio volume. If approved I will implement this myself.
(As a result of discussion below I dropped two out of four proposals and marked them as deleted; but keeping them in the list to preserve their indexes)

1. NVDA+Control+PageUp/PageDown - adjust NVDA volume
2. NVDA+Alt+PageUp/PageDown - adjust volume of all other applications except NVDA.
3. NVDA+alt+s: sound splitter: send NVDA sound to left channel while sound from all other applications to the right channel.
4. NVDA+delete toggle microphone mute.
   I will not insist on these gestures, so feel free to propose other gestures. It would also be fine with me to leave them unassigned.

Utility

Sound split function, initially implemented in Tony's enhancements add-on proved to be very popular among NVDA users, to the extent that it was copied by @josephsl into Sound Splitter add-on and also was copied in NVDA Extension global plugin. On one hand this proves the demand for sound splitting feature; and on the other hand having the same feature implemented in so many add-ons creates incompatibility between them that hurts end users. Therefore I believe it would be benficial to implement this feature in NVDA core. Another reason is that with the recent switch to wasApi, at least two out three add-ons now suffer from compatibility problems, which means it is perfect time to move this feature into NVDA core.
Adjusting the volume of NVDA and other apps separately is closely connected with sound splitting feature: since we diinto two streams, we need to have a way to control their volumes separately.
Toggle microphone mute command proves immensely helpful for people like me who work in large IT companies and have to frequently attend online meetings. At Meta two other screenreader users expressed interest in that command, although they are Jaws users, but this still illustrates the demand.

Technical details

In Tony's enhancements add-on I implemented these commands by using PyCaw library; however if I remember correctly I grabbed a modified version of this library from yet another add-on, name of which I don't remember at this point.
In order to implement this PR, I see a bunch of different approaches with regards to interaction with Audio Manager interface:

1. Add latest version of PyCaw library to the list of dependencies. Drawback: PyCaw seems to rely on a bunch of other libraries.
2. Reuse modified version of PyCaw to NVDA. Drawback: this way seems to be quite hacky.
3. Reuse COM interface ddefinitions from PyCaw but implement all logic from scratch. This seems to be the best option to me at this point.
4. Generate COM interface definition from scratch. I don't have much experience with COM interaction and couldn't figure out how this can be done. I see that NVDA source has a folder comInterfaces that appear to have many COM interfaces seemingly generated in an automatic fashion, but I couldn't figure out how to do that myself.
   At this point I prefer option 3, but I'd like to check with NVAccess if you have different opinion. I wish I could go ahead with option 4, but I spent quite some time trying to generate COM interfaces for MMDevAPI.dll module with no success, so if anyone can teach me how to do so - option 4 would be a great choice then.
   Another detail to call out is the fact that we'd need to create a separate thread to monitor audio sessions. It turns out audio sessions tend to come and go when new applications launch and for example when new system notification sounds play. We need to catch them and apply volume settings as they appear, therefore I implemented a monitor thread that performs this check every second. I haven't found a better solution, so wanted to check with NVAccess if you'd be okay with creating a monitor thread. That thread will only be running when sound split is active.

[Adriani90]

@mltony thank you very much for these great proposals. I think. Regarding NVDA volume, is the volume switcher in the settings ring not already doing the same thing like you propose with nvda+ctrl+page up or down?
Currently the commands for the volume in the settings ring are nvda+ctrl+up and down arrow when focusing the volume category, or nvda+ctrl+shift+up and down arrow on laptop keyboard.

[Adriani90]

for the NVDA+alt+s command, I also propose a checkbox in the newly added audio settings pannel and a combo box which lets you choose if NVDA should go to the right or the left speaker.
Use case:

• If you for example have a headset where you cannot change the left or right earplug, and you need the left ear to hear someone sitting next to you and speaking to you from the left side, it is really inconvenient to have NVDA speaking on the same side. It happened to me very often when using the splitting addon, that's why I decided to have NVDA sound sometimes on the right, sometimes on the left.

[Adriani90]

@mltony not sure about microfone mute gesture, actually this contradicts the philosophy of a screen reader because the microfone as such does not have anything to do with speech output.
However, in Windows 11 at least there is the shortcut key Windows+alt+k which lets you mute the microfone as soon as an application like Teams listens to your microfone. In that case the mute button appears on the taskbar.

Note that there is also the powertoys tool which provides keyboard shorcuts for muting the microfone and turning on and off the camera no matter if you are focusing an application. But i guess not every company would allow you to install powertoys.
https://medium.com/@arba/how-to-mute-your-microphone-or-close-your-camera-on-windows-and-remapped-it-to-a-dedicated-single-5befe2e773ec

To be clear, i think such a global feature to mute the microfone makes totally sense, but i am not sure this is really a feature fitting into a screen reader. Sighted people have the same issue.

[Adriani90]

So I think first of all we should put the focus on the splitting feature and the adjusting of the volume of other apps because this would definitely improve user experience significantly.
And then maybe we can discuss further whether the other two proposals (1. NVDA volume which is already in place and 2. Microfone mute) really need to be implemented or not.

[mltony]

@Adriani90,

1. The commands in the settings ring only affect voice synthesizer. They don't affect volume of other sounds played by NVDA, such as:

• Indentation tones
• Earcons from phonetic punctuation add-on
• Indentation crackling from IndentNav and BrowserNav
• Other sounds from my add-ons, such as SentenceNav, WordNav, BrowserNav.
  I believe a couple of years ago I saw someone complaining on github that there is no such command to adjust volume of all NVDA sounds - but couldn't find that issue now.

2. Agree with NVDA+Alt+S checkbox and combobox.
3. Re microphone mute. I am afraid that talking about philosophy of NVDA is going to be another rabbit hole that I don't want to get into. I can find plenty of examples of commands that don't have anything to do with speech output, for example NVDA+Shift+B reports current battery charge - why do we need that one? Or NVDA+F12 to report time - I mean can't you go to the task bar tray and press review cursor command 12 times to read current time there? The thing is we implement commands as long as they simplify life to a significant degree. At least that's my philosophy in life. Now regarding mute command, I already mentioned some evidence that many working screenreadre users need it. One more argument is that sighted users can typically just click mute button with a mouse in Microsoft teams or whatever application, but for NVDA userse it's going to be quite a few keystrokes until you can get there, so I see direct analogy with NVDA+F12 command. So I strongly disagree that sighted people have the same problem - at least this is not the case with sighted people who work in my company; no sighted person needs a special shortcut to mute microphone because sighties use mouse and that's much more convenient for them. As for Win11 shortcut - you're right there is one, but it appears that there is no notification telling you whether currently microphone is muted or not, so it is useless in its current state. We can try to figure out how to add such a notification, but then the question I'd want to ask would be that it's probably easier to mute/unmute microphone ourselves using API (this proposal), rather than try to find that notification, that might also change in the next versions of Windows. Also my employer is still using Windows 10 and support for Windows 10 is not going away for many years, so it would make sense to build a solution that would work for Win10 - at least for now. The reason why I proposed microphone mute together with other commands here is that it is using the same API and even almost exaclty the same API call - essentially we just adjust volume of another sound device.

[Adriani90]

Thanks Tony for these details, I think your arguments make sense.
As for the volume of NVDA sounds, they can be linked to the voice volume in the audio settings pannel if WASAPI is enabled in advanced settings pannel. however, you are right this might not be the case for sounds comming from addons. I think though that this could be included into that checkbox called "sounds volume follow voice volume". Or is that not feasible?

[Adriani90]

As for the microfone mute, I was thinking more about the uuse case where the application like teams or so is minimized, then sighted people have also to navigate to the application first and then click on mute. That's why I thought it would be kind of strange to have a global command that works even when Teams is minimized. Or is it possible for a sighted person to mute teams directly from another place while it is minimzed?

[CyrilleB79]

Hi Tony

These would be very welcome features.

1. Adjusting "NVDA volume":
   With WASAPI, there are now two volumes: the one of the voice and the one of the sounds. There is also a checkbox to link them. Thus you can check the checkbox and control both volumes together with the synth ring.
   If you wish to improve the UX on this topic, let us know.

2. Adjusting the volumes of other applications:
   I thought that there were an issue related to this but I have not been able to find it.
   This feature may be useful, or alternatively, controlling the volume of the application in foreground.
   IMO, your proposal for a command to control the volume of all applications but NVDA is the best. It's a quick solution when an application in the background is shouting and covering the sound of NVDA's voice.

3. Sound splitting:
   This is already discussed in Audio settings with stereo headset (or speakers) - Send NVDA sounds on one side and the rest of Windows sounds on the other side #12985 (and you are included in the discussion). It has been TRIAGED by NV Access, meaning that they would accept a PR for this. You still may discuss technical details or UX with them (and the community) before implementing it if you wish.
   IMO it's the one to prioritize among the 4 proposals.

4. A shortcut to mute or unmute the mic:
   I am not on Windows 11. But if a shortcut exists on Windows 11, I guess that this feature may remain in a separate add-on. At least it was the type of choices made by NV Access in the past: do not implement a duplicate of a feature already available in Windows; rather make it vocalized if needed. Still, you may implement the feature only for Windows 10 given the number of people remaining on this OS?

Do you plan to implement all these 4 features in the same PR or in separate ones? My opinion is that you should implement the 4 ones separately. This way, you may not hold back, e.g. sound splitting capability while discussing if mute mic should go in NVDA and / or how.
Though, the current issue may be useful if there are shared parts of the design for these features, e.g. using PyCaw library or not...

Cc @jcsteh who has implemented WASAPI.

[josephsl]

Hi,

1. Adjusting audio volume (proposals 1 and 2): you can change app volume using volume mixer, or if you have Windows 11 22H2 Moment 4 or later, press Control+Windows+V to open modern volume mixer from which you can adjust app volume. Even with this, volume mixer deals with volume per app, not with sound output from individual apps (that's adjusted from apps thsmevles).
2. Incorporating Sound Splitter add-on (proposal 3): since the code came from @mltony, I would ask Tony to implement it, although the add-on code may need to be edited to comply with NVDA's own coding style and upgrade to Python 3.11 syntax.
3. Checking mic mute status (proposal 4): on Windows 11, the solution is not audio API, but editing File Explorer app module to let mic mute status be announced from everywhere, a feature I have implemented in Windows App Essentials in the past.

Of these proposals, I think proposal 4 is the easiest to implement although it will be limited to Windows 11 22H2 and later (need to test with Windows 11 21H2 when I have time). Also, if you don't mind, let's discuss one proposal at a time (one issue per proposal please unless a thematic issue/discussion is needed to tie issues together) and refer each proposal issue back to this issue/discussion.

P.S. @LeonarddeR, I know you wanted to limit volume announcement to when File Explorer is focused, I think announcing mic status from everywhere is a good justification for relaxing this a little as it is UIA notification that reports mic status in Windows 11.

Thanks.

[mltony]

@Adriani90 ,
Thanks for the info, I wasn't aware of that checkbox. Then indeed it doesn't make any sense to reimplement it. I updated my proposal - excluded this item.
As for microphone mute, sighted people don't typically minimize application. Therefore they always have a button visible and available within one mouse click. Typical scenario during these meetings is that 90% of time you are muted listening to other people. An when your turn comes - which often times comes all of a sudden, then you need to quickly unmute. Therefore II came up with that shortcut so that my response time is comparable with sighted people.
@CyrilleB79 ,
Re 2: I agree that adjusting the volume of all applications at once is the simpler use case that should probably be enough for like 90% users. The need to control each application separately arises when two or more applications play sounds at the same time. That's not my use case, but I can keep this in mind and make the code reusable in case someone wants to implement this scenario later.
re 3: Ah, that's funny, now I remember, that is your feature request that originally inspired me to write sound splitter in the first place!
Re 4: Fair enough, it seems that @josephsl mentioned he has a solution for automatically announcing microphone mute status. This convinces me that my microphone mute feature should indeed only live in an add-on. I dropped this from my proposal.
@josephsl,
Re 1. I dropped proposal 1. As for proposal 2 (controlling volume of all apps except NVDA) - this is an integral part of sound splitter. When you split sounds into left and right channel, there must be an easy way to control left and right separately, hence this feature.
Re 2. I am @mltony. HI there.
Re 3. Fair enough. I hope announcing of mic mute messages can one day also be merged into NVDA core.
Re ps: At this point I only left two proposals: SoundSplitter and controlling the volume of all other apps. The latter is an integral part of the former. In fact it is impossible to implement sound splitter without chaning the volume of all other apps. So at this point I believe both proposals should stay together in a single issue.

[dreinn]

Hello, Only to say that pageUp and pageDown could be a bit tricky with laptop. For example on my Lenovo keyboard, the pageUp is function-key and down arrow. So for laptop I ask you to use some combinations that can be easier. I'm using the volume adjustment add-on, and for volume it uses ctrl-nvda-horizzontal arrow keys for navigating the app and ctrl-nvda-vertical arrow keys for changing volume. Il giorno sab 13 gen 2024 alle ore 01:55 mltony ***@***.***> ha scritto:

…

@Adriani90 <https://github.com/Adriani90> , Thanks for the info, I wasn't aware of that checkbox. Then indeed it doesn't make any sense to reimplement it. I updated my proposal - excluded this item. As for microphone mute, sighted people don't typically minimize application. Therefore they always have a button visible and available within one mouse click. Typical scenario during these meetings is that 90% of time you are muted listening to other people. An when your turn comes - which often times comes all of a sudden, then you need to quickly unmute. Therefore II came up with that shortcut so that my response time is comparable with sighted people. @CyrilleB79 <https://github.com/CyrilleB79> , Re 2: I agree that adjusting the volume of all applications at once is the simpler use case that should probably be enough for like 90% users. The need to control each application separately arises when two or more applications play sounds at the same time. That's not my use case, but I can keep this in mind and make the code reusable in case someone wants to implement this scenario later. re 3: Ah, that's funny, now I remember, that is your feature request that originally inspired me to write sound splitter in the first place! Re 4: Fair enough, it seems that @josephsl <https://github.com/josephsl> mentioned he has a solution for automatically announcing microphone mute status. This convinces me that my microphone mute feature should indeed only live in an add-on. I dropped this from my proposal. @josephsl <https://github.com/josephsl>, Re 1. I dropped proposal 1. As for proposal 2 (controlling volume of all apps except NVDA) - this is an integral part of sound splitter. When you split sounds into left and right channel, there must be an easy way to control left and right separately, hence this feature. Re 2. I am @mltony <https://github.com/mltony>. HI there. Re 3. Fair enough. I hope announcing of mic mute messages can one day also be merged into NVDA core. Re ps: At this point I only left two proposals: SoundSplitter and controlling the volume of all other apps. The latter is an integral part of the former. In fact it is impossible to implement sound splitter without chaning the volume of all other apps. So at this point I believe both proposals should stay together in a single issue. — Reply to this email directly, view it on GitHub <#16037 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5H6GD6NBP6PYBHCMWK3MTYOHLRHAVCNFSM6AAAAABBYVX4VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJQGIZDANBVGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Simone Dal Maso www.nvapple.it www.nvda.it

[Adriani90]

I think in this case you will have to use a tool to remap some of the keys you don‘t use to page up and down. There are also other commands in NVDA which use these keys such as changing review mode, scrolling by paage etc. I still stay with tony‘s approach. We cannot consider every keyboard configuration out there. We should stick to the standard keyboard configuration. People with non standard keyboard configs have to remap coresponding keys.There is also an NVDA add on for remapping keys I think. Von meinem iPhone gesendetAm 13.01.2024 um 19:47 schrieb dreinn ***@***.***>: Hello, Only to say that pageUp and pageDown could be a bit tricky with laptop. For example on my Lenovo keyboard, the pageUp is function-key and down arrow. So for laptop I ask you to use some combinations that can be easier. I'm using the volume adjustment add-on, and for volume it uses ctrl-nvda-horizzontal arrow keys for navigating the app and ctrl-nvda-vertical arrow keys for changing volume. Il giorno sab 13 gen 2024 alle ore 01:55 mltony ***@***.***> ha scritto:

…

@Adriani90 <https://github.com/Adriani90> , Thanks for the info, I wasn't aware of that checkbox. Then indeed it doesn't make any sense to reimplement it. I updated my proposal - excluded this item. As for microphone mute, sighted people don't typically minimize application. Therefore they always have a button visible and available within one mouse click. Typical scenario during these meetings is that 90% of time you are muted listening to other people. An when your turn comes - which often times comes all of a sudden, then you need to quickly unmute. Therefore II came up with that shortcut so that my response time is comparable with sighted people. @CyrilleB79 <https://github.com/CyrilleB79> , Re 2: I agree that adjusting the volume of all applications at once is the simpler use case that should probably be enough for like 90% users. The need to control each application separately arises when two or more applications play sounds at the same time. That's not my use case, but I can keep this in mind and make the code reusable in case someone wants to implement this scenario later. re 3: Ah, that's funny, now I remember, that is your feature request that originally inspired me to write sound splitter in the first place! Re 4: Fair enough, it seems that @josephsl <https://github.com/josephsl> mentioned he has a solution for automatically announcing microphone mute status. This convinces me that my microphone mute feature should indeed only live in an add-on. I dropped this from my proposal. @josephsl <https://github.com/josephsl>, Re 1. I dropped proposal 1. As for proposal 2 (controlling volume of all apps except NVDA) - this is an integral part of sound splitter. When you split sounds into left and right channel, there must be an easy way to control left and right separately, hence this feature. Re 2. I am @mltony <https://github.com/mltony>. HI there. Re 3. Fair enough. I hope announcing of mic mute messages can one day also be merged into NVDA core. Re ps: At this point I only left two proposals: SoundSplitter and controlling the volume of all other apps. The latter is an integral part of the former. In fact it is impossible to implement sound splitter without chaning the volume of all other apps. So at this point I believe both proposals should stay together in a single issue. — Reply to this email directly, view it on GitHub <#16037 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5H6GD6NBP6PYBHCMWK3MTYOHLRHAVCNFSM6AAAAABBYVX4VCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJQGIZDANBVGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Simone Dal Maso www.nvapple.it www.nvda.it —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[josephsl]

Hi,

Update on mic mute proposal: Windows+Alt+K is ineffective - supposedly only Microsoft Teams supports this, and turns out Teams does not respond to Windows+Alt+K shortcut (when I create a meeting in Teams and Zoom, both apps do not respond to Windows+Alt+K shortcut but provides their own hotkeys to mute/unmute the microphone). Discord had a feature request to support Windows+Alt+K shortcut. Unless an app that does respond to Windows+Alt+K is found, I think microphone toggle may need to be considered part of this proposal or a separate one. Also, do note that some vendors include a command to toggle microphone on their computers (for example, ThinkPad series uses Fn+F4).

Thanks.

[dpy013]

hello @mltony
Here's a similar plugin for NVDA
https://github.com/huaiyinfeilong/audiomanager
You can use it as a reference to improve your PR

[seanbudd]

Hi @mltony - please split the 2 remaining feature request proposals into separate issues so that they can be triaged, discussed and fixed separately.