Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android: Expandable/Collapsible State #30841

Closed
amarlette opened this issue Feb 3, 2021 · 4 comments
Closed

Android: Expandable/Collapsible State #30841

amarlette opened this issue Feb 3, 2021 · 4 comments

Comments

@amarlette
Copy link

amarlette commented Feb 3, 2021

Description

The "expanded" accessibilityState for Android is currently faked by appending to the content description, rather than implemented correctly. This has all sorts of side effects, such as:

  • On some components, the state expanded/collapsed is properly announced on focus, on some it is not.
  • On some components only the expanded/collapsed state is announced, and not other component text.
  • Upon change, state change is not always announced.
  • The accessibilityState's "expanded" field does not seem to work on all element types (for example, it has no effect on 's).
  • using accessibilityActions it is possible to add an action for expand/collapse, but these are treated as custom actions and must have their own label defined, rather than using Androids built in expand/collapse actions, which Talkback has predefined labels for.

React Native version:

v0.63

Expected Behavior

Upon focus, either "expanded" or "collapsed" should be announced, depending on whether the accessibilityState's "expanded" field is set to true or false. Upon state change, the new state should be announced. If an expand/collapse action is desired, you should be able to use the default system ones, rather than having to define custom ones yourself.

Snack

https://snack.expo.io/0YOQfXFBi

Android Details

Expandable and Collapsible are unique in the Android Accessibility API, in that they are not represented as properties on the View or AccessibilityNodeInfo, but are only represented as AccessibilityActions on the AccessibilityNodeInfo. This means that Talkback determines whether or not a node is "expandable" or "collapsible", or potentially even both, by looking at the list of AccessibilityActions attached to the AccessibilityNodeInfo.

When setting the accessibilityState's expandable property, it should correlate to adding an action of either AccessibilityNodeInfoCompat.ACTION_EXPAND or AccessibilityNodeInfoCompat.ACTION_COLLAPSE on the AccessibilityNodeInfo. This work should be done in the ReactAccessibilityDelegate class's

Currently, this feature is being "faked" by appending to the contentDescription in the BaseViewManager class. This should be removed when this feature is implemented properly.

@grgr-dkrk
Copy link
Contributor

I'll try it.

@ta-cos
Copy link

ta-cos commented Jan 5, 2022

@amarlette @blavalla @kacieb @grgr-dkrk

Hello all, I would like to take this one on... if @grgr-dkrk is no longer working on it, if i am able to take it on, could someone, asign it to me?

@fabOnReact
Copy link
Contributor

I'm preparing a solution for this issue. I tested the solution proposed by the author of the Issues, and it seems to work.

@fabOnReact fabOnReact self-assigned this Aug 25, 2022
@fabOnReact
Copy link
Contributor

fabOnReact commented Aug 26, 2022

Related to the issue Main Branch - onAccessibilityAction not called on Fabric renderer detected on the main branch and not related to this issue.

Systrace.startAsyncFlow(
Systrace.TRACE_TAG_REACT_JAVA_BRIDGE, event.getEventName(), event.getUniqueID());
}

E/unknown:ReactEventEmitter( 3845): com.facebook.react.bridge.ReactNoCrashSoftException: 
Cannot find EventEmitter for receiveEvent: SurfaceId[1] ReactTag[104] UIManagerType[2] EventName[topAccessibilityAction]
E/unknown:ReactEventEmitter( 3845):     
at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
E/unknown:ReactEventEmitter( 3845):     
at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:228)
full log

E/unknown:ReactEventEmitter( 6121):     at com.facebook.react.uimanager.events.ReactEventEmitter.receiveEvent(ReactEventEmitter.java:151)
E/unknown:ReactEventEmitter( 6121):     at com.facebook.react.uimanager.events.Event.dispatchModern(Event.java:229)
E/unknown:ReactEventEmitter( 6121):     at com.facebook.react.uimanager.events.EventDispatcherImpl$DispatchEventsRunnable.run(EventDispatcherImpl.java:370)
E/unknown:ReactEventEmitter( 6121):     at android.os.Handler.handleCallback(Handler.java:938)
E/unknown:ReactEventEmitter( 6121):     at android.os.Handler.dispatchMessage(Handler.java:99)
E/unknown:ReactEventEmitter( 6121):     at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
E/unknown:ReactEventEmitter( 6121):     at android.os.Looper.loopOnce(Looper.java:201)
E/unknown:ReactEventEmitter( 6121):     at android.os.Looper.loop(Looper.java:288)
E/unknown:ReactEventEmitter( 6121):     at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:228)
E/unknown:ReactEventEmitter( 6121):     at java.lang.Thread.run(Thread.java:1012)

public void receiveEvent(
int surfaceId,
int targetReactTag,
String eventName,
boolean canCoalesceEvent,
int customCoalesceKey,
@Nullable WritableMap event,
@EventCategoryDef int category) {
@UIManagerType int uiManagerType = ViewUtil.getUIManagerType(targetReactTag);
if (uiManagerType == UIManagerType.FABRIC && mFabricEventEmitter != null) {
mFabricEventEmitter.receiveEvent(

fabOnReact added a commit to fabOnReact/react-native that referenced this issue Nov 29, 2022
ReactAccessibilityDelegate performAccessibilityAction dispatches the event topAccessibilityAction
--
Understand why Fabric does not call registerEventEmitter before performAccessibilityAction for that reactTag/surfaceId
Read StackTrace from error message (use printStacktrace)
The method getUIManager retrieve the UIManager from the reactTag and does not check if is Fabric or Paper

ReactAccessibilityDelegate [performAccessibilityAction](https://github.com/facebook/react-native/blob/bf37a34c38b39a14de8194520d07de0b9f8c0bf7/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L410-L415) dispatches the event topAccessibilityAction
Understand why Fabric does not call [registerEventEmitter](https://github.com/fabriziobertoglio1987/react-native/blob/163171ccab6937785f4f3c85e011bd14540bebf5/ReactAndroid/src/main/java/com/facebook/react/fabric/FabricUIManager.java#L426) before performAccessibilityAction for that reactTag/surfaceId
Read StackTrace from [error message](facebook#30841 (comment)) (use [printStacktrace](https://www.google.com/search?q=java+printStacktrace))
The method [getUIManager](https://github.com/fabriziobertoglio1987/react-native/blob/dc4c54ec1b7b7f7cd37c7402b1eac5292cb4996a/ReactAndroid/src/main/java/com/facebook/react/uimanager/UIManagerHelper.java#L40-L48) retrieve the UIManager from the reactTag and does not check if is Fabric or Paper
facebook-github-bot pushed a commit that referenced this issue Dec 9, 2022
Summary:
fixes #30841 (comment). onAccessibilityAction does not work on Fabric and logs:

```
E/unknown:ReactEventEmitter( 3845): com.facebook.react.bridge.ReactNoCrashSoftException:
Cannot find EventEmitter for receiveEvent: SurfaceId[1] ReactTag[104] UIManagerType[2]
```

## Changelog

[Android] [Fixed] - Fix onAccessibilityAction on Fabric

Pull Request resolved: #35507

Test Plan: #35507 (comment)

Reviewed By: javache

Differential Revision: D41707777

Pulled By: philIip

fbshipit-source-id: 0f4550a17f4b8bfc1aefa404059b367907f8f60d
OlimpiaZurek pushed a commit to OlimpiaZurek/react-native that referenced this issue May 22, 2023
…/Collapsible State (facebook#34353)

Summary:
>Expandable and Collapsible are unique in the Android Accessibility API, in that they are not represented as properties on the View or AccessibilityNodeInfo, but are only represented as AccessibilityActions on the AccessibilityNodeInfo. This means that Talkback determines whether or not a node is "expandable" or "collapsible", or potentially even both, by looking at the list of AccessibilityActions attached to the AccessibilityNodeInfo.

>When setting the accessibilityState's expandable property, it should correlate to adding an action of either AccessibilityNodeInfoCompat.ACTION_EXPAND or AccessibilityNodeInfoCompat.ACTION_COLLAPSE on the AccessibilityNodeInfo. This work should be done in the ReactAccessibilityDelegate class's

>Currently, this feature is being "faked" by appending to the contentDescription in the BaseViewManager class. This should be removed when this feature is implemented properly.

fixes facebook#30841

## Changelog

[Android] [Fixed] - using AccessibilityNodeInfo#addAction to announce Expandable/Collapsible State

Pull Request resolved: facebook#34353

Test Plan:
- On some components, the state expanded/collapsed is properly announced on focus, on some it is not.
- On some components only the expanded/collapsed state is announced, and not other component text.
- Upon change, state change is not always announced.
- The accessibilityState's "expanded" field does not seem to work on all element types (for example, it has no effect on 's).
- using accessibilityActions it is possible to add an action for expand/collapse, but these are treated as custom actions and must have their own label defined, rather than using Androids built in expand/collapse actions, which Talkback has predefined labels for.

https://snack.expo.io/0YOQfXFBi

Tests  15th August 2022:
- Paper [Tests](facebook#34353 (comment))
- Fabric [Tests](facebook#34353 (comment))

Tests 6th September 2022:
- [Button which keeps control of extended/collapsed state in JavaScript with onAccessibilityAction, accessibilityActions and accessibiltyState (Paper)](facebook#34353 (comment))
- [TouchableWithoutFeedback keeps control of extended/collapsed state in Android Widget (Paper)](facebook#34353 (comment))
- [TouchableWithoutFeedback keeps control of extended/collapsed state in Android Widget (Fabric)](facebook#34353 (comment))
- [TouchableOpacity announces visible text and triggers expanded/collapsed with onPress and accessiblity menu (Fabric)](facebook#34353 (comment))

Announcing state with custom actions on Fabric (FAIL).
The issue is not a regression from this PR, as documented in facebook#34353 (comment). It will be fixed in a separate PR.

Reviewed By: NickGerleman

Differential Revision: D39893863

Pulled By: blavalla

fbshipit-source-id: f6af78b1839ba7d97eca052bd258faae00cbd27b
OlimpiaZurek pushed a commit to OlimpiaZurek/react-native that referenced this issue May 22, 2023
Summary:
fixes facebook#30841 (comment). onAccessibilityAction does not work on Fabric and logs:

```
E/unknown:ReactEventEmitter( 3845): com.facebook.react.bridge.ReactNoCrashSoftException:
Cannot find EventEmitter for receiveEvent: SurfaceId[1] ReactTag[104] UIManagerType[2]
```

## Changelog

[Android] [Fixed] - Fix onAccessibilityAction on Fabric

Pull Request resolved: facebook#35507

Test Plan: facebook#35507 (comment)

Reviewed By: javache

Differential Revision: D41707777

Pulled By: philIip

fbshipit-source-id: 0f4550a17f4b8bfc1aefa404059b367907f8f60d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment