Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override default Talkback automatic content grouping and generate a custom contentDescription #33690

Closed
wants to merge 61 commits into from

Commits on Apr 6, 2022

  1. draft

    fabOnReact committed Apr 6, 2022
    Configuration menu
    Copy the full SHA
    05a2bc3 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2022

  1. draft

    fabOnReact committed Apr 11, 2022
    Configuration menu
    Copy the full SHA
    4ffd15d View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2022

  1. Configuration menu
    Copy the full SHA
    5e1622f View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2022

  1. draft

    fabOnReact committed Apr 14, 2022
    Configuration menu
    Copy the full SHA
    28f86a3 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2022

  1. Configuration menu
    Copy the full SHA
    963cd17 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2022

  1. Configuration menu
    Copy the full SHA
    69473de View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bdabc4c View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2022

  1. draft commit

    fabOnReact committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    ae66001 View commit details
    Browse the repository at this point in the history
  2. remove getRole logic

    fabOnReact committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    7c287a4 View commit details
    Browse the repository at this point in the history
  3. adding comments to methods

    fabOnReact committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    6d73cd4 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    2a3a836 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    982fb20 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2022

  1. Conditions that should trigger the custom contentDescription

    facebook#33690 (comment)
    
    The custom contentDescription should be triggered if:
    - an element has no contentDescription **AND** text
    - **AND** has other content to announce (role, state, etc.)
    fabOnReact committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    e0e1629 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a22c435 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2022

  1. Configuration menu
    Copy the full SHA
    07bac60 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    821e623 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    35190c0 View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2022

  1. Configuration menu
    Copy the full SHA
    986b10b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    28ed277 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e1521cf View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    90dd4a8 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    5e7aaf3 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    1f06997 View commit details
    Browse the repository at this point in the history
  7. Returns a cached instance if such is available or a new one is create.

    The returned instance is initialized from the given <code>info</code>.
    fabOnReact committed Apr 27, 2022
    Configuration menu
    Copy the full SHA
    a05b912 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2022

  1. remove not required logic

    - node.recycle does not trigger runtime as we use
      obtain(AccessibilityNodeInfo info)
    - setContentDescription(null) will use the default content description
      (Button, Text number 1, Text number 2) which is the existing behaviour
    fabOnReact committed Apr 28, 2022
    Configuration menu
    Copy the full SHA
    64bb48a View commit details
    Browse the repository at this point in the history

Commits on May 24, 2022

  1. Configuration menu
    Copy the full SHA
    7cf8fce View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0b620e0 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2022

  1. check for isScreenReaderFocusable and hasHint

    adding check on hintText in hasText method
    the "hint text" for EditText's also counts as content, as it's still announced when the EditText has no text or contentDescription.
    See Talkbacks implementation here for reference:
    https://github.com/google/talkback/blob/6c0b475b7f52469e309e51bfcc13de58f18176ff/utils/src/main/java/com/google/android/accessibility/utils/AccessibilityNodeInfoUtils.java#L1663
    facebook#33690 (comment)
    
    adding check for isScreenReaderFocusable in isAccessibilityFocusable
    See Talkback's implementation here for reference:
    https://github.com/google/talkback/blob/6c0b475b7f52469e309e51bfcc13de58f18176ff/utils/src/main/java/com/google/android/accessibility/utils/AccessibilityNodeInfoUtils.java#L1690
    facebook#33690 (comment)
    fabOnReact committed May 26, 2022
    Configuration menu
    Copy the full SHA
    45e3040 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5488aba View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    751b83e View commit details
    Browse the repository at this point in the history
  4. add addStateSegments to custom announcement

    The previous developed functionality would remove the following
    announcements from the child nodes:
    - state
    - disabled
    - accessibilityRole (for the child)
    
    The fields were not retrieved/added to the new Talkback announcement
    Fixes regression tested in example
    751b83e
    
    the current announcement have been added, the other will follow up in
    the next commits:
    - state
    fabOnReact committed May 26, 2022
    Configuration menu
    Copy the full SHA
    e3edefb View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a0fb221 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2022

  1. Configuration menu
    Copy the full SHA
    d2c8bfb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    334b24d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9d72a70 View commit details
    Browse the repository at this point in the history
  4. EditText child component

    fabOnReact committed May 27, 2022
    Configuration menu
    Copy the full SHA
    9d1bf23 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2022

  1. Configuration menu
    Copy the full SHA
    abed89e View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2022

  1. Configuration menu
    Copy the full SHA
    b90fff4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f1852f1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6bd06c4 View commit details
    Browse the repository at this point in the history
  4. fix circleci tests

    https://app.circleci.com/pipelines/github/facebook/react-native/13665/workflows/efdceb92-2020-4ca4-8317-f3889b0b7800/jobs/258498
    
    ```
    Cannot create `TouchableNativeFeedback` element because property `style` is missing in `Props` [1] but exists in
    props [2]. [prop-missing]
    
       packages/rn-tester/js/examples/Accessibility/AccessibilityExample.js:264:16
       264|               <TouchableNativeFeedback
                           ^^^^^^^^^^^^^^^^^^^^^^^
    
    References:
       Libraries/Components/Touchable/TouchableNativeFeedback.js:97:55
        97| class TouchableNativeFeedback extends React.Component<Props, State> {
                                                                  ^^^^^ [1]
       packages/rn-tester/js/examples/Accessibility/AccessibilityExample.js:264:15
                          v-----------------------
       264|               <TouchableNativeFeedback
       265|                 style={{
       266|                   height: 100,
       267|                   width: 100,
       268|                   backgroundColor: 'yellow',
       269|                 }}
       270|                 focusable={true}
       271|                 onPress={() => console.warn('onPress child')}
       272|                 accessible={false}
       273|                 accessibilityLabel="this is my label"
       274|                 accessibilityRole="image"
       275|                 accessibilityState={{disabled: true}}
       276|                 accessibilityValue={{text: 'this is the accessibility value'}}>
       277|                 <Text accessible={false}>Text number 3</Text>
       278|               </TouchableNativeFeedback>
                          -------------------------^ [2]
    
    Found 1 error
    error Command failed with exit code 2.
    info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
    Failed to run flow.
    Finished.
    /home/circleci/react-native/scripts/run-ci-javascript-tests.js:47
        throw Error(exitCode);
        ^
    
    Error: 1
        at Object.<anonymous> (/home/circleci/react-native/scripts/run-ci-javascript-tests.js:47:11)
        at Module._compile (node:internal/modules/cjs/loader:1103:14)
        at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
        at Module.load (node:internal/modules/cjs/loader:981:32)
        at Function.Module._load (node:internal/modules/cjs/loader:822:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
        at node:internal/main/run_main_module:17:47
    
    Exited with code exit status 1
    ```
    fabOnReact committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    2dc702e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8622c13 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e4b31c3 View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2022

  1. Revert "moving getRoleDescription to another branch"

    This reverts commit 8622c13.
    fabOnReact committed Jun 2, 2022
    Configuration menu
    Copy the full SHA
    251df27 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2022

  1. Configuration menu
    Copy the full SHA
    8aa5b25 View commit details
    Browse the repository at this point in the history
  2. removing accessibilityRole functionality as not required from specs

    if the child has a role defined, it should probably also be accessible on its own,
      and therefore won't get pulled up into its parents announcement in the first place.
    facebook#30719 (comment)
    fabOnReact committed Jun 3, 2022
    Configuration menu
    Copy the full SHA
    b804bd0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f77a4a2 View commit details
    Browse the repository at this point in the history
  4. Moving not required logic to a separate branch

    based on my personal notes from the last meeting and the below comment
    facebook#30719 (comment)
    fabOnReact committed Jun 3, 2022
    Configuration menu
    Copy the full SHA
    44122ff View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2022

  1. Configuration menu
    Copy the full SHA
    cb0e1b6 View commit details
    Browse the repository at this point in the history
  2. adding test cases scenarios

    Scenarios:
    - 3 levels of nested components 10min
    - the contentDescription is not empty and does not have nodeText
    - contentDescription is not empty and is not an EditText
    - contentDescription is empty, is an EditText and has a nodeText
    fabOnReact committed Jun 6, 2022
    Configuration menu
    Copy the full SHA
    0c79fc8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    46826d8 View commit details
    Browse the repository at this point in the history
  4. fix flow circleci error

    fabOnReact committed Jun 6, 2022
    Configuration menu
    Copy the full SHA
    ad6732a View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2022

  1. Configuration menu
    Copy the full SHA
    d5f4c9c View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2022

  1. One of the child has accessibilityState (hasStateDescription triggers…

    … the announcement)
    
    from facebook#33690 (comment)
    
    I noticed that getStateDescription returns null even when adding accessibilityState. Calling setStateDescription will update the state description, which is not set via the react accessibilityState or other props.
    
    Seems that the method [setViewState](https://github.com/fabriziobertoglio1987/react-native/blob/ad6732aa3576786d37478a729b112031dd94b682/ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManager.java#L254) does not call setStateDescription.
    
    I will further investigate tomorrow.
    
    <details><summary>View#setStateDescription</summary>
    <p>
    
    ```java
        /**
         * Sets the {@link View}'s state description.
         * <p>
         * A state description briefly describes the states of the view and is primarily used
         * for accessibility support to determine how the states of a view should be presented to
         * the user. It is a supplement to the boolean states (for example, checked/unchecked) and
         * it is used for customized state description (for example, "wifi, connected, three bars").
         * State description changes frequently while content description should change less often.
         * State description should be localized. For android widgets which have default state
         * descriptions, app developers can call this method to override the state descriptions.
         * Setting state description to null restores the default behavior.
         *
         * @param stateDescription The state description.
         * @see #getStateDescription()
         */
        @RemotableViewMethod
        public void setStateDescription(@nullable CharSequence stateDescription) {
            if (mStateDescription == null) {
                if (stateDescription == null) {
                    return;
                }
            } else if (mStateDescription.equals(stateDescription)) {
                return;
            }
            mStateDescription = stateDescription;
            if (!TextUtils.isEmpty(stateDescription)
                    && getImportantForAccessibility() == IMPORTANT_FOR_ACCESSIBILITY_AUTO) {
                setImportantForAccessibility(IMPORTANT_FOR_ACCESSIBILITY_YES);
            }
            if (AccessibilityManager.getInstance(mContext).isEnabled()) {
                AccessibilityEvent event = AccessibilityEvent.obtain();
                event.setEventType(AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED);
                event.setContentChangeTypes(AccessibilityEvent.CONTENT_CHANGE_TYPE_STATE_DESCRIPTION);
                sendAccessibilityEventUnchecked(event);
            }
        }
    ```
    
    </p>
    </details>
    
    This is the implementation of TalkBack [hasStateDescription](https://github.com/google/talkback/blob/6c0b475b7f52469e309e51bfcc13de58f18176ff/utils/src/main/java/com/google/android/accessibility/utils/AccessibilityNodeInfoUtils.java#L1672-L1677).
    
    The following fields of the view are set with the accessibilityState:
    
    - [checkable](https://github.com/fabriziobertoglio1987/react-native/blob/ad6732aa3576786d37478a729b112031dd94b682/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L411) => BOOLEAN_PROPERTY_CHECKABLE
    - [enabled](https://github.com/fabriziobertoglio1987/react-native/blob/ad6732aa3576786d37478a729b112031dd94b682/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L408) => BOOLEAN_PROPERTY_ENABLED
    
    <details><summary>setCheckable</summary>
    <p>
    
    https://github.com/aosp-mirror/platform_frameworks_base/blob/19e53cfdc8a5c6ef45c0adf2dd239576ddce5822/core/java/android/view/accessibility/AccessibilityNodeInfo.java#L2008
    
    ```java
      /**
       * Sets whether this node is checkable.
       * <p>
       *   <strong>Note:</strong> Cannot be called from an
       *   {@link android.accessibilityservice.AccessibilityService}.
       *   This class is made immutable before being delivered to an AccessibilityService.
       * </p>
       *
       * @param checkable True if the node is checkable.
       *
       * @throws IllegalStateException If called from an AccessibilityService.
       */
      public void setCheckable(boolean checkable) {
          setBooleanProperty(BOOLEAN_PROPERTY_CHECKABLE, checkable);
      }
    ```
    
    </p>
    </details>
    
    <details><summary>setEnabled</summary>
    <p>
    
    https://github.com/aosp-mirror/platform_frameworks_base/blob/19e53cfdc8a5c6ef45c0adf2dd239576ddce5822/core/java/android/view/accessibility/AccessibilityNodeInfo.java#L2227
    
    ```java
        /**
         * Sets whether this node is enabled.
         * <p>
         *   <strong>Note:</strong> Cannot be called from an
         *   {@link android.accessibilityservice.AccessibilityService}.
         *   This class is made immutable before being delivered to an AccessibilityService.
         * </p>
         *
         * @param enabled True if the node is enabled.
         *
         * @throws IllegalStateException If called from an AccessibilityService.
         */
        public void setEnabled(boolean enabled) {
            setBooleanProperty(BOOLEAN_PROPERTY_ENABLED, enabled);
        }
    ```
    
    </p>
    </details>
    
    The implementation of `hasStateDescription` is still valid
    fabOnReact committed Jun 8, 2022
    Configuration menu
    Copy the full SHA
    c17f00e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    087e642 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    51f23f0 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8b84521 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2022

  1. Configuration menu
    Copy the full SHA
    dc0cc5a View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2022

  1. Configuration menu
    Copy the full SHA
    eac460e View commit details
    Browse the repository at this point in the history
  2. updating examples in AccessibilityExample

    This example feels a bit odd to me. The state itself is not concatenated into the announcement, which makes sense, as it would lead to potentially announcing conflicting state if two child elements had separate states like "checked" and "unchecked", but because of this I'm not sure what value this example is providing. If there was no state here and just the label it would work the exact same way.
    I think we can probably just remove this one and maybe replace it with one where the parent element has a state and making sure that state is reflected even when the children are combined into it.
    facebook#33690 (comment)
    facebook#33690 (comment)
    
    I don't think importantForAccessibility="yes" is necessary for any of these examples, so lets remove it on all of them just to make it clear that this isn't a prequalification for this feature working.
    facebook#33690 (comment)
    
    Remove duplicate example (this is the same as the previous one)
    facebook#33690 (comment)
    fabOnReact committed Aug 30, 2022
    Configuration menu
    Copy the full SHA
    ddfb345 View commit details
    Browse the repository at this point in the history
  3. improving examples titles

    fabOnReact committed Aug 30, 2022
    Configuration menu
    Copy the full SHA
    11c4b00 View commit details
    Browse the repository at this point in the history