Making links independently focusable by Talkback

[fabOnReact]

Summary

This issue fixes 32004. The Pull Request was previously published by blavalla with 31757.

This is a follow-up on D23553222, which made links functional by using Talkback's Links menu. We don't often use this as the sole access point for links due to it being more difficult for users to navigate to and easy for users to miss if they don't listen to the full description, including the hint text that announces that links are available.
The Implementation of the functionality consists of:

Retrieving the accessibility links and triggering the TalkBack Focus over the Text

1. nested Text components with accessibilityRole link are saved as ReactClickableSpan instances in Android native TextView (more info)
2. If the TextView contains any ClickableSpans (which are nested Text components with role link), set a view tag and reset the accessibility delegate.
3. Obtain each link description, start, end, and position relative to the parent Text (id) from the Span as an AccessibilityLink
4. Use the AccessibilityLink to display TalkBack focus over the link with the getVirtualViewAt method (more info)

Implementing ExploreByTouchHelper to detect touches over links and to display TalkBack rectangle around them.

1. ReactAccessibilityDelegate inherits from ExploreByTouchHelper
2. If the ReactTextView has an accessibility delegate, trigger ExploreByTouchHelper method dispatchHoverEvent
3. Implements the methods getVirtualViewAt and onPopulateBoundsForVirtualView.
   The two methods implements the following functionalities (more info):
   • detecting the TalkBack onPress/focus on nested Text with accessibilityRole="link"
   • displaying TalkBack rectangle around nested Text with accessibilityRole="link"

Changelog

[Android] [Added] - Make links independently focusable by Talkback

Test Plan

1. User Interacts with links through TalkBack default accessibility menu (link)
2. The nested link becomes the next focusable element after the parent element that contains it. (link)
3. Testing accessibility examples in pr branch (link)
4. Testing accessibility android examples in pr branch (link)
7. TalkBack focus moves through links in the correct order from top to bottom (PR Branch with link.id) (link to video test) (discussion)
8. TalkBack focus does not move through links in the correct order from top to bottom (PR Branch without link.id) (link to video test) (discussion)

Test on main branch
5. Testing accessibility examples in main branch (link)
6. Testing accessibility android examples in main branch (link)

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
ios	-	universal	n/a	--

Base commit: 97f3eb2
Branch: main

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	8,211,273	+5,691
android	hermes	armeabi-v7a	7,819,965	+5,683
android	hermes	x86	8,583,000	+5,685
android	hermes	x86_64	8,533,395	+5,677
android	jsc	arm64-v8a	9,878,879	+4,492
android	jsc	armeabi-v7a	8,872,132	+4,509
android	jsc	x86	9,846,699	+4,493
android	jsc	x86_64	10,441,231	+4,490

Base commit: 97f3eb2
Branch: main

[blavalla]

Just one comment to verify, but overall it looks good to me.

[blavalla]

Can you verify that this is still necessary? At the time this was written, if this didn't happen focus would move to the last link in the paragraph on the first swipe, and then the next swipe would start navigating through the links from the bottom to the top. I'm not sure if this is still necessary or not, as it seemed like it may have been an unintentional side effect of the way we create these ClickableSpans, and that code may have changed since then.

A video showing a focus on a paragraph and then a swipe through the links both with this code and without it should be enough.

This may also be impacting the order that the links appear in the "Links" menu, although their order there isn't as important, as it's out of the context of the text itself. Ideally these would still match though and the first link in the text would be first in the list as well as the first swipe.

[fabOnReact]

Can you verify that this is still necessary? At the time this was written, if this didn't happen focus would move to the last link in the paragraph on the first swipe, and then the next swipe would start navigating through the links from the bottom to the top. I'm not sure if this is still necessary or not, as it seemed like it may have been an unintentional side effect of the way we create these ClickableSpans, and that code may have changed since then.

Thanks @blavalla Yes. It is still necessary.

This may also be impacting the order that the links appear in the "Links" menu, although their order there isn't as important, as it's out of the context of the text itself. Ideally these would still match though and the first link in the text would be first in the list as well as the first swipe.

I run a test of the functionality on the main branch and the PR branch.

PR branch	Main branch

I remain available for improvements. Thanks

A video showing a focus on a paragraph and then a swipe through the links both with this code and without it should be enough.

I included below the test cases and updated the summary of the Pull Request.

TalkBack focus moves through links IN THE CORRECT ORDER from top to bottom (PR Branch with link.id)

Testing with the link.id in AccessibilityLink (discussion)

// ID is the reverse of what is expected, since the ClickableSpans are returned in reverse
// order due to being added in reverse order. If we don't do this, focus will move to the
// last link first and move backwards.
//
// If this approach becomes unreliable, we should instead look at their start position and
// order them manually.
link.id = spans.length - 1 - i;
links.add(link);

Expected Result:
Swiping move TalkBack focus in this order:

1. Parent Text This is a test of inline links in React Native. Here's another link. Here is a link that spans multiple lines because the text is so long. This sentence has no links in it. links available
2. Nested Texts in order from top to bottom (test, inline links, another link, link that spans multiple lines...)

Links are displayed in the above order in the TalkBack menu

Actual Results:

RESULT 1 (SUCCESS) - Swiping moves TalkBack focus in the correct order

1. Parent Text This is a test of inline links in React Native. Here's another link. Here is a link that spans multiple lines because the text is so long. This sentence has no links in it. links available
2. Nested Texts in order from top to bottom

2022-03-07.09-11-19.mp4

RESULT 2 (FAIL) - Links are NOT displayed in the correct order in the TalkBack menu

2022-03-07.09-13-23.mp4

TalkBack focus does NOT move through links in the correct order from top to bottom (PR Branch without link.id)

Testing without the link.id in AccessibilityLink (discussion)

// ID is the reverse of what is expected, since the ClickableSpans are returned in reverse
// order due to being added in reverse order. If we don't do this, focus will move to the
// last link first and move backwards.
//
// If this approach becomes unreliable, we should instead look at their start position and
// order them manually.
// link.id = spans.length - 1 - i;
links.add(link);

Expected Result:
Swiping move TalkBack focus in this order:

1. Parent Text This is a test of inline links in React Native. Here's another link. Here is a link that spans multiple lines because the text is so long. This sentence has no links in it. links available
2. Nested Texts in order from top to bottom (test, inline links, another link, link that spans multiple lines...)

Links are displayed in the above order in the TalkBack menu

Actual Results:

RESULT 1 (FAIL) - Swiping moves TalkBack focus in this wrong order

1. Parent Text This is a test of inline links in React Native. Here's another link. Here is a link that spans multiple lines because the text is so long. This sentence has no links in it. links available
2. Swipe right or down moves the focus to the last link link that spans multiple lines because the text is too long.

2022-03-07.08-07-55.mp4

RESULT 2 (FAIL) - Links are NOT displayed in the correct order in the TalkBack menu

2022-03-07.08-17-20.mp4

[facebook-github-bot]

@philIip has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[react-native-bot]

This pull request was successfully merged by @fabriziobertoglio1987 in 7b5b114.

^{When will my fix make it into a release? | Upcoming Releases}