Add screen reader announcement to last block in list

[alexstine]

Is your feature request related to a problem? Please describe.

When in navigation mode, you do not know how many blocks are in the list and can sometimes get in to other meta boxes with one tab too many.

Steps:

1. Open Gutenberg.
2. Switch to navigation mode.
3. Use the tab key to tab through the blocks.
4. Once you reach the last block, press tab again.
5. If plugins, themes, or mu-plugins have added extra meta boxes, you will land there and be a bit disoriented.

Describe the solution you'd like

I would like each tab press to announce which block you are on. E.g. "3 out of 5".

Describe alternatives you've considered

I thought about just having the last block have a special "last block in list" announcement, but it would be more practical to list the range "current block through max."

Thoughts?

[talldan]

Sounds like a good idea. The Block Navigation / List View feature has a similar description when navigating through blocks:
https://github.com/WordPress/gutenberg/blob/master/packages/block-editor/src/components/block-navigation/utils.js

[afercia]

Played a bit with this and maybe something like getBlockCount() could be used to get the number of blocks. Each block already has a position prop that is currently used to build the string with the "row" number. By using these two things, building a string that communicates the position and total count seems relatively simple.

Each "block-selection-button" should thus announce something like this:

• Block 1 of 10
• Block 2 of 10
• etc.
• Block 10 of 10

Worth reminding there's also the case when multiple blocks are placed horizontally and the string announces the "Column" number.

I'd tend to think the current term "Row" isn't ideal and announcing "n of n" would be better. I'd keep the column announcement for horizontal blocks.

However, I think all these labels should be reviewed and changed. Currently, they use an aria-label attribute that contains also the whole textual content of a block, or other info for non-textual blocks (e.g. "Empty" for image blocks with no image set).

While this helps screen reader users to understand the content of a block, it totally breaks the expected behavior for other assistive technology users e.g. speech recognition software users. For example:

• visually, the button displays the text "Paragraph"
• the button aria label is actually something like "Paragraph Block. Row 2. plus the whole block textual content"
• the text presented visually and the accessible name mismatch, violating WCAG 2.1 Label in Name
• speech recognition software users will try to voice a command like "Click Paragraph" because "Paragraph" is the visible text
• the command may fail because the actual accessible name is different
• also, translations will likely have the words used to build the current string in a different order, for example:
  • the first part of the aria-label is "Paragraph Block"
  • in languages other than English, these two words are often translated in a different order (at least in many European languages), for example in Italian that would be "Blocco Paragrafo" so there's a mismatch even without the additional info about the Row and block textual content

To remediate to this, I think there's the need to change these aria-labels implementation and:

• make sure the visible text and the aria-label (if needed) always match
• move the additional info to a separate, visually hidden, element referenced by aria-describedby
• this element should contain the info about the following things:
  • the block position and total number of blocks (n of n)
  • additional info for some specific blocks e.g.: headings level (for example, the current string isn't that ideal: "Heading Block. Row 2. Level 2. My heading")
  • the block textual content (or, maybe better, an excerpt of the content) or other info for non-textual blocks

The order of this various pieces in the description should be considered with care to provide the needed info in the most understandable, efficient, way.

/Cc @MarcoZehe

[MarcoZehe]

However, I think all these labels should be reviewed and changed. Currently, they use an aria-label attribute that contains also the whole textual content of a block, or other info for non-textual blocks (e.g. "Empty" for image blocks with no image set).

Yes, and that was done on purpose. Blind users have no efficient way to get an overview of the content otherwise, without adding a cillion required key presses. Efficiency is the key point here, we simply don't have the benefit of an overview, or skimming over/glancing at, several points on the screen at once. Speech is a sequential medium, and so is braille. So combining the most important points into a concise label that gives us all the required information at once, when browsing the blocks, is a key efficiency requirement.

While this helps screen reader users to understand the content of a block, it totally breaks the expected behavior for other assistive technology users e.g. speech recognition software users. For example:

I realize that this isn't ideal for those using speech recognition. Apple have overcome this in native iOS apps by adding another property to accessibility objects that contains a more concise label, and which Voice Control in iOS 13 and above uses to find and activate items if told so via the Voice Control interface. VoiceOver users never see (or rather, hear) this more concise label, preserving compatibility.

However, I believe Apple have not yet exposed this kind of thing via Safari, and thus there is no attempt at specifying such an alternative label for voice activation yet.

I also think that the WCAG criteria doesn't take into account complex widgets such as Gutenberg at all, where having custom labels is a key factor for efficient use, see above. For example, if we were to go with the visual label which only says "Paragraph", there would be no way to know which paragraph this is. With 20 paragraphs or so, adding two or three extra keystrokes to get to the actual content to know which paragraph is being referenced, would add 60 keystrokes.

To remediate to this, I think there's the need to change these aria-labels implementation and:

• make sure the visible text and the aria-label (if needed) always match

• move the additional info to a separate, visually hidden, element referenced by aria-describedby

• this element should contain the info about the following things:

  • the block position and total number of blocks (n of n)
  • additional info for some specific blocks e.g.: headings level (for example, the current string isn't that ideal: "Heading Block. Row 2. Level 2. My heading")
  • the block textual content (or, maybe better, an excerpt of the content) or other info for non-textual blocks

aria-describedby, or the acc description in general, for that matter, is additional, supplemental information, that is not guaranteed to be read by screen readers. There are often even settings to disable reading of the acc description because historically, the title attribute was read as this additional info, and still is, but many users wanted a way to turn this off because of the way web developers have abused this title attribute for decades. And even if it is read, it is always the last element spoken. So the information that is important comes after the label, type, and state of an element have been read.

On MacOS, such information is always read with a delay, because there, it is mapped to help text. So VoiceOver users will either have to shorten that delay to 0, or for each block, wait for the default of 2 seconds before that additional info is read which they may not be interested in for other elements. That is 40 seconds wasted when navigating 20 blocks.

The order of this various pieces in the description should be considered with care to provide the needed info in the most understandable, efficient, way.

That I agree on. But I am very very concerned that users may actually totally miss this information if it is moved to the supplemental, secondary, description.

[afercia]

@MarcoZehe thanks for your feedback. Yep, I do know this was made on purpose, that's why I pinged you in my previous comment 🙂

Regarding this point:

I also think that the WCAG criteria doesn't take into account complex widgets such as Gutenberg at all, where having custom labels is a key factor for efficient use, see above. For example, if we were to go with the visual label which only says "Paragraph", there would be no way to know which paragraph this is. With 20 paragraphs or so, adding two or three extra keystrokes to get to the actual content to know which paragraph is being referenced, would add 60 keystrokes.

To clarify: the problem is that there is already a visual label which only says "Paragraph". When the block editor is in "navigation mode" the buttons screen reader users get when navigating through the blocks only display the block type. So for example, the buttons will show the following text:

• Paragraph
• Heading
• Image
• etc.

SInce there's a mismatch between the visible text and the actual accessible name, activating these buttons with a speech recognition software will likely fail. Testing with macOS Voice Control I can't activate a heading button voicing "Click Heading". To make it work, I have to voice the following command:

"Click Heading block row 2"

where "row 2" is the block position that is included in the aria-label. This string is not exposed visually therefore sighted speech recognition software users can't activate these buttons with a direct voice command. I can't test with Dragon naturally speaking on Windows but I guess it's the same issue there as well.

It's interesting that iOS added another property to accessibility objects for use with Voice Control but I'm afraid that doesn't help us. We need a solution based on standards that works across any assistive technology.

I'm still not convinced that the current implementation is correct. The aria-label isn't really meant to contain dozens, potentially hundreds of words. In our case, it also needs to match what is presented visually. Or, at least, the text exposed visually needs to match the first part of the aria-label.

Worth also mentioning that, when a block has a very long content, the aria-live message that announces the switch to navigation mode is spoken only after the entire block content contained in the aria-label has been announced. The content could be so long that many users might just stop the screen reader and miss the aria-live message.

Overall, I think we should explore a better solution. I'd greatly appreciate to take advantage from your expertise to find together a more standard way to communicate all the necessary info to all users, whether they're screen reader users or users of other assistive technologies. Any thoughts more than welcome!

[afercia]

In a way, #25126 has a similar issue as in: there's the need to make additional information available to assistive technology users. Trying to recap the current situation:

• aria-label is currently used in a non-compliant way: it is meant to be used to communicate the name of a control and instead it brings in the whole block content
• while this may be okay for some screen reader users, it harms other assistive technologies users, for example: speech recognition software users
• using aria-describedby is not an option as the description may not be announced depending on a variety of factors including the user verbosity setting
• some assistive technology implemented proprietary mechanisms to announce additional information but this isn't a standard

Still, we need a reliable way to provide users with additional information concerning the block, for example:

• the block content or part of it so that users can understand which block is selected
• additional actions, for example: in the case of nested blocks (Columns block, Group block and the like) which keys can be used to move through blocks of the same level and through nested blocks

GIven there's no standard solution to this, what if Gutenberg implements its own solution? From a conversation with @mariohamann, one option could be a keyboard shortcut that works only when the block button is focused. For example:

• the block button is focused
• the block button aria-label only reports the block type and position e.g.: "Paragraph block, 5 of 10"
• but, the aria-label contains one more bit of text, for example: "Press {keyboard shortcut here} for detailed info"
• the keyboard shortcut triggers an aria-live message with all the additional information we'd want to provide

Maybe this mechanism could work well and I like the fact that the potentially huge amount of additional information is communicated only on user request. A downside is that it would require a keyboard shortcut... bringing in all the problems related to potential conflicts, see #3218.

I'll propose the accessibility team to discuss this issue in the next weekly meetings.

[alexstine]

@afercia These are my thoughts.

using aria-describedby is not an option as the description may not be announced depending on a variety of factors including the user verbosity setting

I disagree. I think aria-describedby would be a fine application for this. My reasoning is if users are advanced enough to change verbosity settings, they should be advanced enough to understand what verbosity in this context could do. In Mac Voiceover, there is a way to actually customize verbosity and other Voiceover settings based on a specific URL so they could keep normal verbosity in the editor while they could have their desired verbosity every where else.

In my opinion making things accessible should be making it work for the largest amount of users possible. I use aria-describedby in a lot of projects I have contributed to and I have never received any complaints from users/other screen reader testers that they could not get the contextual information due to a verbosity setting.

I am really against keyboard shortcuts because there are just too many of them and if you are not in the right mode, Browse vs. Forms/Edit mode, it could have some adverse effects. In my opinion, it would be easier to use aria-describedby and not have to put so much time in to educating users about keyboard shortcuts.

I do support the idea of custom shortcuts. Besides everyone has their own way of making info accessible to them, I especially see shortcuts being useful for enabling Bold, Italics, etc. I just don't know if I completely support the idea of a shortcut to read out the block text or any additional information that may be included.

There is also this principal of less key strokes with programs such as JAWS. JAWS also operates better for one handed users due to the wrap to top feature. I kind of agree on the same philosophy. The less key strokes to find contextual information, the better.

Anyway, some of these points are a bit out there, but do you have any follow-up thoughts?

Thanks.

[afercia]

@alexstine thanks for your thoughts.

To be clear, I was initially in favor of using aria-describedby because, as you pointed out, it's simpler. More importantly: it's standard. Only after @MarcoZehe's remarks I tried to consider an alternative solution.

At this point it seems there's some disagreement between screen reader users too 🙂

In my limited experience with screen readers as tester, I never faced problems with aria-describedby. I can understand the verbosity argument but as you pointed out this can be tweaked by users. I'd just like to reach some consensus to that we ca move on with this issue 🙂

I'd just like to note again that the current implementation makes the aria-label contain too many words, potentially hundreds of words and this is something I see as non-compliant and harmful for other assistive technology users.

[alexstine]

@MarcoZehe How would you like to move this forward? Thinking about this now, block position is not the most important thing to announce to users if for some reason they have a low verbosity setting. My opinion is we can't cover every edge case, all we can do is make it work for 90% of users and we've done our job. If those other 10% of screen reader users have a super low verbosity setting, they could always add an exception, in the case of Mac Voiceover, and then it will only read aria-describedby on their WordPress website URL.

Attempting to move this forward...

Thanks.

[MarcoZehe]

Go with aria-describedby. An extra keyboard shortcut is less discoverable.

[alexstine]

Hmm, I need to get back to this one. I went off on a tangent trying to fix a much bigger issue and never landed the change. I'll circle back here after I get some free time, hopefully soon.