Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS/Android: Text input error for screenreaders #12

Closed
fabOnReact opened this issue Mar 4, 2022 · 64 comments
Closed

iOS/Android: Text input error for screenreaders #12

fabOnReact opened this issue Mar 4, 2022 · 64 comments

Comments

@fabOnReact

This comment was marked as outdated.

@fabOnReact
Copy link
Owner Author

fix issue with missing drawable

with drawable without drawable

@fabOnReact
Copy link
Owner Author

fabOnReact commented Mar 18, 2022

Change of error state from onChangeText show/hides a TextInput error

Expected Result:
The Error shows next to the TextInput

Actual Result:
No Error Show next to the TextInput

Applying changes from with facebook/react-native#29070 fixes the issue.

click to open test

function ErrorExample(): React.Node {
  const [text, setText] = React.useState('');
  const [error, setError] = React.useState(null);
  const textinput = React.useRef(null);
  return (
    <>
      <Button title="create error" onPress={() => setError('button')} />
      <TextInput
        ref={textinput}
        errorMessage={error}
        onBlur={() => setError('onBlur')}
        onChangeText={newText => {
          setText(newText);
          setError(newText === 'error' ? 'this input is invalid' : null);
        }}
        value={text}
      />
    </>
  );
}

without PR 29070

without facebook/react-native#29070

without_pr_29070.mp4
with PR 29070

with facebook/react-native#29070

with_pr_29070.mp4

@fabOnReact
Copy link
Owner Author

fabOnReact commented Mar 18, 2022

Test Cases of the functionality (Android - Fabric)

error added onChangeText

2022-04-21.17-03-38.mp4

error added on onEndEditing

2022-04-21.17-04-45.mp4

@fabOnReact

This comment was marked as resolved.

fabOnReact added a commit to fabOnReact/react-native that referenced this issue Mar 23, 2022
allow to display error message on TextInput triggered in a onChangeText
callback. Fixes issue explained in:
fabOnReact/react-native-notes#12 (comment)

Related links:
fabOnReact/react-native-notes#12 (comment)
d3d54e1
fabOnReact added a commit to fabOnReact/react-native that referenced this issue Mar 23, 2022
allow to display error message on TextInput triggered in a onChangeText
callback. Fixes issue explained in:
fabOnReact/react-native-notes#12 (comment)

Related links:
fabOnReact/react-native-notes#12 (comment)
d3d54e1
fabOnReact added a commit to fabOnReact/react-native that referenced this issue Mar 23, 2022
allow to display error message on TextInput triggered in a onChangeText
callback. Fixes issue explained in:
fabOnReact/react-native-notes#12 (comment)

Related links:
fabOnReact/react-native-notes#12 (comment)
d3d54e1
@fabOnReact
Copy link
Owner Author

fabOnReact commented Mar 24, 2022

AndroidTextInputProps and AndroidTextInputState.cpp

https://github.com/fabriziobertoglio1987/react-native/blob/a7a781ff4a13e744f4eb3007ef0657740b277a72/ReactCommon/react/renderer/components/textinput/androidtextinput/react/renderer/components/androidtextinput/AndroidTextInputProps.cpp#L268

fabOnReact/react-native@5abe584

ReactEditText - need to add the errorMessageAndroid to AndroidTextInputState
currently errorMessageAndroid is part of the props but not of the state, while underlineColorAndroid is part of state.

image

I added errorMessageAndroid prop to

https://github.com/fabriziobertoglio1987/react-native/blob/b5786376f4efc590386860a574d29db90f3cddaf/ReactCommon/react/renderer/components/textinput/androidtextinput/react/renderer/components/androidtextinput/AndroidTextInputProps.cpp#L67-L70

but needs to be added also to AndroidTextInputState

https://github.com/fabriziobertoglio1987/react-native/blob/b5786376f4efc590386860a574d29db90f3cddaf/ReactCommon/react/renderer/components/textinput/androidtextinput/react/renderer/components/androidtextinput/AndroidTextInputState.cpp#L29

so the StateWrapper

https://github.com/fabriziobertoglio1987/react-native/blob/b5786376f4efc590386860a574d29db90f3cddaf/ReactAndroid/src/main/java/com/facebook/react/uimanager/StateWrapper.java#L4

https://github.com/fabriziobertoglio1987/react-native/blob/b5786376f4efc590386860a574d29db90f3cddaf/ReactAndroid/src/main/java/com/facebook/react/views/textinput/ReactTextInputManager.java#L1291

includes also the errorMessageAndroid

need to follow similar implementation of underlineColorAndroid

https://github.com/fabriziobertoglio1987/react-native/commits/681ed402de9d6f119cd30583ad5048e1cb75bab4/ReactCommon/react/renderer/components/textinput/androidtextinput/react/renderer/components/androidtextinput/AndroidTextInputState.cpp

@fabOnReact

This comment was marked as resolved.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Mar 25, 2022

passing mock values to AndroidTextInputShadowNode.cpp

to pass mock value of errorMessageAndroid to the constructor, I would need to generate a string of type jsi::String

https://github.com/fabriziobertoglio1987/react-native-notes/blob/67b466f083cd32066dc289a4eaf65b1151d97e10/ReactCommon/react/renderer/components/textinput/androidtextinput/react/renderer/components/androidtextinput/AndroidTextInputShadowNode.cpp#L155-L166

The method createFromAscii could be used to generate a random string.

Relevant links:

Compilation error messages

image

@fabOnReact

This comment was marked as resolved.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Mar 25, 2022

see facebook/react-native#33468 (comment)

TextInputState paragraphAttributes and textAttributes

image

The above state is added here

https://github.com/fabriziobertoglio1987/react-native-notes/blob/b5786376f4efc590386860a574d29db90f3cddaf/ReactCommon/react/renderer/components/textinput/androidtextinput/react/renderer/components/androidtextinput/AndroidTextInputState.cpp#L85-L88

The functionality was initially introduced by adding the StateParagraph component commit 06ce568 to the state.

Updates the paragraph component to use State instead of Local Data, part of the path to a Fabric TextInput 💯

and then commit fabOnReact/react-native@13db6cb

This diff extends ParagraphState to expose not only the AttributedString associated to Text components, but also ParagraphAttributes that describes the visual high level props of the Paragraph

The AndroidTextInputState seems composed of two components:

  • paragraph attributes like ellipsizemode textBreakStrategy maximumNumberOfLines
    This attribute control the styling
  • attributesString in which we should add the errorMessageAndroid to update the state

https://github.com/fabriziobertoglio1987/react-native-notes/blob/b5786376f4efc590386860a574d29db90f3cddaf/ReactCommon/react/renderer/components/text/ParagraphState.h#L29-L38

Debug Statements fabOnReact/react-native@1e54f76

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as outdated.

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as resolved.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Mar 30, 2022

setting the TextInput errorMessage state with setTextAndSelection Java API from JavaScript

Related links

Expected Result:
An error message is displayed on the TextInput when the user types error in the field

Actual Result:
No error message is displayed. The state update triggered in JavaScript is prior to the fabric state updates which over-rides the state and removes the message

click to open code used in the test

function ErrorExample(): React.Node {
  const [text, setText] = React.useState('');
  const [error, setError] = React.useState(null);
  return (
    <>
      <Button onPress={() => setError('button')} title="Press to set error" />
      <Text>
        Type error in the below TextInput to display an error message.
      </Text>
      <TextInput
        maximumNumberOfLines={2}
        accessibilityRole="button"
        hyphenationFrequency="normal"
        errorMessageAndroid={error}
        onBlur={() => setError('onBlur')}
        onEndEditing={() => setError('onEndEditing')}
        onChangeText={newText => {
          setText(newText);
          if (newText === 'error') {
            setError('this input is invalid');
          } else if (error !== 'onBlur') {
            setError(null);
          }
        }}
        value={text}
        style={[styles.default, {height: 50}]}
      />
    </>
  );
}

click to open recording of test case

2022-03-30.11-49-06.mp4

@fabOnReact

This comment was marked as off-topic.

@fabOnReact
Copy link
Owner Author

Test Cases of the functionality (Paper) after removing changes to .cpp libs
2022-05-12.19-45-17.mp4

@fabOnReact
Copy link
Owner Author

fabOnReact commented May 12, 2022

xcode stuck when loading pods

My MacBook Pro has very limited storage (125GB model). I spend considerable amount of time saving storage and did the mistake of enabling iCloud. The files were stored on iCloud and xcode was having issues downloading them.
I'm buying new MacBook Pro this weekend 🥳 .

I also experienced this on my new MacBook Pro, I enabled iCloud by mistake.

Screen.Recording.2022-05-12.at.20.53.42.mov

@fabOnReact

This comment was marked as resolved.

@fabOnReact

This comment was marked as outdated.

@fabOnReact

This comment was marked as outdated.

@fabOnReact
Copy link
Owner Author

fabOnReact commented May 20, 2022

setMostRecentEventCount is called after invoking setScreeenreaderError

https://github.com/fabriziobertoglio1987/react-native/blob/4e8af270100916b8b67198020b402a02020de8d9/Libraries/Components/TextInput/TextInput.js#L1143-L1159

_onChange callback does the following:

  1. saves the currentText from event.nativeEvent.text. This is the text received from native side (iOS or Android)
  2. calls onChangeText(currentText).
    Here the onChangeText javascript callback may take the currentText from native and change it.
    The native and javascript version of currentText may differ here.
  3. call setLastNativeText(currentText) which saves in state the last value of the text sent from the native side
  4. increase the mostRecentEventCount

useLayoutEffect does the following:

  1. if lastNativeText (from iOS/Android) differs from the props.value
  2. we update the lastNativeText to props.value
  3. we trigger setTextAndSelection with props.value to over-ride the native iOS/Android value of the text
    This callback runs only after onChangeText, as mostRecentEventCount is increased once onChangeText code is executed.

With the screenreaderError we don't receive a value from native (iOS or Android) that we want to over-ride.
The screenreaderError is sent from JavaScript as prop. The native value should allways be over-ridden from the JavaScript value.

I think the issue here is we use setTextAndSelection to over-ride the native value (iOS, Android) of the text, but this over-rides also something else.
The other solution was adding to the fabric TextInputState field ParagraphState the field screenreaderError.
When c++ and android/ios send each other updates about the changes in state, the change in the screenreaderError is detected (onChangeText) and the value is displayed, otherwise the update is not shown.
I decided to not implement this solution because as the problem is common between iOS and Android, and the update is triggered from onChangeText in JavaScript. Seems to me that the solution should be done in JavaScript.

I run a test with another prop textAlign. textAlign is will update if changed onChangeText callback and it is included ReactTextUpdate https://github.com/facebook/react-native/search?q=textAlign

fabOnReact added a commit to fabOnReact/react-native that referenced this issue May 20, 2022
reapply the previous solution after doing a reset of the branch
changes deleted are saved in branch

https://github.com/facebook/react-native/compare/main...fabriziobertoglio1987:text-input-errors-set-text-and-selection?expand=1

Moving the head back to this commit
facebook@29b99ba

Reapply the solution for iOS announcing multiple times the errorMessage
facebook@e0bdf61
facebook@3467c5d

A similar problem was also detected on Android and solved with a similar
solution
fabOnReact/react-native-notes#12 (comment)

The reset was done to remove the changes to the implementation which
would use codegen.setTextAndSelection() instead of adding the prop to
c++

Explained in detail here
fabOnReact/react-native-notes#12 (comment)

we have 2 types of data flow in the TextInput

SCENARIO 1
Javascript  <--->  Native Android/iOS

JavaScript receives in callback like onChangeText updates on the value
of the text and selection position, then decides with an onLayoutEffect
callback to over-ride those updates with the value kept in internal
state. This is how JavaScript TextInput component works.

SCENARIO 2
CPP <----> Native Android/iOS

The state in CPP/Android/iOS keeps in sync.

In this case, we don't receive any native update from Android/iOS on the
value of the screenreaderError in Native side (not like for example the
text value which is passed back to JavaScript with the _onChange
callback).
https://github.com/fabriziobertoglio1987/react-native/blob/f2e23215ca14c3c630aa931cdd114187589ac0fb/Libraries/Components/TextInput/TextInput.js#L1115

For this reason, I don't think is a good idea including the
screenreaderError in the setTextAndSelection.
@fabOnReact
Copy link
Owner Author

iOS - announcing error onChangeText and screenreader focus

HEAD fabOnReact/react-native@bb4597e

new.test.low.quality.mp4

@fabOnReact

This comment was marked as resolved.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 9, 2022

iOS - The screenreader announces the TextInput value after the errorMessage is cleared

source code for example

function ErrorExample(): React.Node {
  const [text, setText] = React.useState('');
  const [error, setError] = React.useState(null);
  const [accessibilityInvalid, setAccessibilityInvalid] = React.useState(false);
  return (
    <>
      <Button onPress={() => setError('button')} title="Press to set error" />
      <Text>
        Type error in the below TextInput to display an error message.
      </Text>
      <TextInput
        accessibilityErrorMessage={error}
        accessibilityInvalid={accessibilityInvalid}
        // onBlur={() => setError('onBlur')}
        // onEndEditing={() => setError('onEndEditing')}
        onChangeText={newText => {
          setText(newText);
          if (newText === 'Error') {
            setError('the newText is: ' + newText);
            setAccessibilityInvalid(true);
          } else if (newText === 'empty') {
            setError(newText);
            setAccessibilityInvalid(true);
          } else if (newText === 'null') {
            setError('');
            setAccessibilityInvalid(false);
          } else {
            setError(newText);
            setAccessibilityInvalid(false);
          }
        }}
        value={text}
        style={styles.default}
      />
    </>
  );
}

Video of the test

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 10, 2022

iOS - Exception thrown while executing UI block: - [RCTTextView setOnAccessibiltyAction:]: unrecognized selector sent to instance (Paper) (main branch)

Issue experienced on main branch when opening the accessibility example. The second error displays Exception thrown while executing UI block: 'parentNode' is a required parameter if we remove the accessibilityActions and onAccessibilityAction from the AccessibilityExample.
Solution https://stackoverflow.com/a/61570223/7295772

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 13, 2022

iOS - Exception thrown while executing UI block: - RCTUITextView setAccessibilityErrorMessage:]: unrecognized selector sent to instance (iOS - Paper on main branch)

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 13, 2022

iOS - accessibilityErrorMessage is not announced on Paper

  • The current solution does work on Paper
  • The TextInput component is based on UITextInput
  • The method updateProps is not called on Paper, but works on Fabric

Steps to solve this problem:

I was able to implement a solution in TextInput.js to trigger the accessibility announcement in JavaScript.
The announcement works correctly onChangeText (Fabric and Paper), but on Paper does not announce the error outside of the onChangeText callback.

  • try to trigger the announcement manually in JS when the prop updates on Paper
  • on Fabric this diffing logic triggers the error. Verify that similar diffing logic is triggered on Paper (check shadow node).

The implementation of accessibilityValue and accessibilityValueInternal should be enough for this functionality and I believe no native code needs to be added for iOS.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 14, 2022

Fabric and Paper use different implementations to handle diffing for props. Changes to native iOS implementation should be minimal, as the accessibilityValue prop is already implemented for iOS.
In Fabric the correct implementation is:

  1. The accessibilityValue is first set with the accessibilityValue setter
  2. accessibilityValue updates are done in RCTViewComponent. The previous solution (trigger manually the notification with UIAccessibilityPostNotification) could fix the issue of errors being announced two times. Currently this issue does not reproduce if we don't set the accessibilityValue and just trigger the announcement in javascript, but then screenreader does not announce the error onHover.

In Paper the accessibilityValue is set:

I was able to trigger the announcement correctly onChangeText (no on hover) using only JavaScript:

  1. adding useEffect callback in TextInput.js
  React.useEffect(() => {
    // triggers the error onChangeText
    if (props.accessibilityInvalid === true) {
      AccessibilityInfo.announceForAccessibilityWithOptions(
        props.accessibilityErrorMessage,
        {queue: true},
      );
    }
  }, [props.accessibilityInvalid, props.accessibilityErrorMessage]);
  1. Not passing the accessibilityErrorMessage to iOS RCTTextInputView
    The above solution does not announce the error when the screenreader focus moves to the TextInput. The Screenreader needs to read the accessibilityValue, but setting this value may cause announcing the error two times.

@fabOnReact

This comment was marked as resolved.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 15, 2022

Implementation of TextInput Component on iOS

RCTBaseTextInputView is a wrapper View around the UITextField:
setAttributedText is part of RCTBaseTextInputView.m.

RCTBaseTextInputView does not implement backedTextInputView which needs to be over-ridden in RCTSinglelineTextInputView.

  1. It modifies the TextInput component text and textAttributes using backedTextInputView
  2. backedTextInputView (RCTUITextField) is a wrapper around UITextField

RCTUITextField is a iOS Component build on top of UITextField:
The setter setAccessibilityErrorMessage is part of Singleline/RCTUITextField, which inherits methods/functionalities from RCTBackedTextInputViewProtocol and UITextField

Solution:

  1. save the prop accessibilityErrorMessage in an instance variable of Singleline/RCTUITextField
  2. update the accessibilityValue when the text is set with RCTBaseTextInputView#setAttributedText
  3. set the accessibilityValue to the text + self.backedTextInputView.accessibilityErrorMessage

Issues:

  1. updating the instance variable self.errorMessage from RCTUITextField does not display any change in RCTBaseTextInputView#setAttributedText. The error is always null.

Todo:

  • RCTUITextField#setAccessibilityErrorMessage does not update self.accessibilityErrorMessage
    RCTUITextField#setAccessibilityErrorMessage can update self.accessibilityValue
    Compare the implementations of self.accessibilityValue and self.accessibilityErrorMessage
  • self.backedTextInput.accessibilityErrorMessage = "error" works in setAttributedText. Try to log the values in setAttributedText to verify it.
  • verify that iOS Fabric announces error outside of onChangeText
  • fix tests cases from commit facebook/react-native@582b1e8 also on Fabric
  • use same iOS Paper solution on Fabric
  • verify iOS Paper solution works with non controlled TextInput
  • comments in my own code review
  • test multiline TextInput on iOS
  • sometimes when the RN App builds, the screenreader announces random letter.. for example "o"
    but I don't think is a concern, because the changes are scoped for TextInput Component

@fabOnReact

This comment was marked as off-topic.

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 17, 2022

iOS - announce lastChar (not entire text) onChangeText and avoid multiple announcements (Fabric)

Commit fabOnReact/react-native@b53ed9d fixes the following issues on iOS Fabric (apply solution from Paper)

1)The error is announced, but not with the character onChangeText
2)The error is announced multiple times

Related #12 (comment) #12 (comment)

sourcecode

function AccessibilityErrorWithButtons(): React.Node {
  const [text, setText] = React.useState('');
  const [error, setError] = React.useState(null);
  const [accessibilityInvalid, setAccessibilityInvalid] = React.useState(false);
  return (
    <View>
      <RNTesterBlock>
        <TextInput
          accessibilityErrorMessage={error}
          accessibilityInvalid={accessibilityInvalid}
          onChangeText={newText => {
            if (newText === 'Error') {
              setError('this is the error');
              setAccessibilityInvalid(true);
            } else {
              setError('');
              setAccessibilityInvalid(false);
            }
            setText(newText);
          }}
          value={text}
          style={styles.default}
        />
        <Button
          onPress={() => {
            setError('This error is set with Button onPress callback');
          }}
          title="Press to set an accessibilityErrorMessage to a non-empty value"
        />
        <Button
          onPress={() => setAccessibilityInvalid(!accessibilityInvalid)}
          title={`Press to set accessibilityInvalid to ${
            accessibilityInvalid ? 'false' : 'true'
          }`}
        />
        <Button
          onPress={() => setError('')}
          title={`Press to set an empty error message`}
        />
      </RNTesterBlock>
    </View>
  );
}

low_quality.mp4

fabOnReact added a commit to fabOnReact/react-native that referenced this issue Jun 17, 2022
1)The error is announced, but not with the character onChangeText
2)The error is announced multiple times

Test Case of this changes
fabOnReact/react-native-notes#12 (comment)
@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 17, 2022

iOS - announces or does not announce the accessibilityError through Button onPress (not onChangeText) (Fabric)

sourcecode

function AccessibilityErrorWithButtons(): React.Node {
  const [text, setText] = React.useState('');
  const [error, setError] = React.useState(null);
  const [accessibilityInvalid, setAccessibilityInvalid] = React.useState(false);
  return (
    <View>
      <RNTesterBlock>
        <TextInput
          accessibilityErrorMessage={error}
          accessibilityInvalid={accessibilityInvalid}
          onChangeText={newText => {
            if (newText === 'Error') {
              setError('this is the error');
              setAccessibilityInvalid(true);
            } else {
              setError('');
              setAccessibilityInvalid(false);
            }
            setText(newText);
          }}
          value={text}
          style={styles.default}
        />
        <Button
          onPress={() => {
            setError('This error is set with Button onPress callback');
          }}
          title="Press to set an accessibilityErrorMessage to a non-empty value"
        />
        <Button
          onPress={() => setAccessibilityInvalid(!accessibilityInvalid)}
          title={`Press to set accessibilityInvalid to ${
            accessibilityInvalid ? 'false' : 'true'
          }`}
        />
        <Button
          onPress={() => setError('')}
          title={`Press to set an empty error message`}
        />
      </RNTesterBlock>
    </View>
  );
}

first_low.mp4

@fabOnReact
Copy link
Owner Author

fabOnReact commented Jun 17, 2022

iOS - the error is announced with accessibilityInvalid true and does not clear after typing text (onChangeText) (Fabric)

sourcecode

function AccessibilityErrorDoesNotClear(): React.Node {
  const [text, setText] = React.useState('');
  const [error, setError] = React.useState('');
  const [accessibilityInvalid, setAccessibilityInvalid] = React.useState(false);
  return (
    <View>
      <RNTesterBlock title="TextView without label">
        <Text>Set an error, change textinput value and then announce it.</Text>
        <TextInput
          accessibilityErrorMessage={error}
          accessibilityInvalid={accessibilityInvalid}
          value={text}
          onChangeText={newText => setText(newText)}
          style={styles.default}
        />
        <Text>
          The value of the accessibilityErrorMessage prop is{' '}
          {error.length > 0 ? error : 'empty error'}
        </Text>
        <Button
          onPress={() => {
            setError('This is an error');
          }}
          title="sets accessibilityErrorMessage to 'This is an error'"
        />
        <Button
          onPress={() => setAccessibilityInvalid(!accessibilityInvalid)}
          title={`Press to set accessibilityInvalid to ${
            accessibilityInvalid ? 'false' : 'true'
          }`}
        />
      </RNTesterBlock>
    </View>
  );
}

second_low.mp4

facebook-github-bot pushed a commit to facebook/react-native that referenced this issue Mar 1, 2023
Summary:
**Android**: The functionality consists of calling the [AccessibilityNodeInfo#setError][10] and [#setContentInvalid][13] method to display the error message in the TextInput.

**Fixes [https://github.com/facebook/react-native/issues/30848][51] - Adding an accessibilityErrorMessage prop to the TextInput Component**:
**Android**: The prop accessibilityErrorMessage triggers the AccessibilityNodeInfo method [setError][10] which automatically sets the correct properties on the AccessibilityNodeInfo that will inform screen readers of this state. The method calls setContentInvalid(true) and setError(youErrorString) on the AccessibilityNodeInfo.

**Fixes [https://github.com/facebook/react-native/issues/30859][52] -  Detecting changes in the Error state (text inputs)**
**Fabric - Android** - Adding accessibilityErrorMessage to field AndroidTextInputState.
ReactTextInputManager and ReactEditText receive state updates both from [Javascript][32] and [cpp (fabric)][34].
- accessibilityErrorMessage is added to the fabric AndroidTextInputState field
- The updates are received in the ReactAndroid API with method updateState from ReactTextInputManager
- After updating the TextInput text with onChangeText, the update of the accessibilityErrorMessage is triggered with method maybeSetAccessibilityError which triggers [setError][10].

More info:
- An explanation of [state updates between fabric and ReactAndroid for the TextInput component][34]
- [ReactNative renderer state updates][35]

**Paper - Android** - Adding accessibilityErrorMessage to ReactTextInputShadowNode to trigger updates in Paper renderer when accessibilityErrorMessage is changed within the onChange callback.

Related Links (Android):
- [In this diff I'm shipping and deleting mapBufferSerialization for Text measurement][101]
- [This diff implement and integrates Mapbuffer into Fabric text measure system][39]
- [Refactor ViewPropsMapBuffer -> general MapBuffer props mechanism][100]
- [TextInput: support modifying TextInputs with multiple Fragments (Cxx side)][24]
- [TextInput: keep C++ state in-sync with updated AttributedStrings in Java][23]
- [AccessibilityNodeInfo#setError][11]
- [Explanation on how TextInput calls SET_TEXT_AND_SELECTION in Java API][32]
- [Fabric: convertRawProp was extended to accept an optional default value][27]
- [understanding onChangeText callback][31]
- [Editable method replace()][12]
- [Change of error state from onChangeText show/hides a TextInput error][30]
- [AndroidTextInput: support using commands instead of setNativeProps (native change)][25]
- [TextInput: support editing completely empty TextInputs][26]
- [[Android] Fix letters duplication when using autoCapitalize https://github.com/facebook/react-native/issues/29070][40]
- [Support optional types for C++ TurboModules][28]
- [discussion on using announceForAccessibility in ReactEditText][36]
- [ fix annoucement delayed to next character][61]
- [Announce accessibility state changes happening in the background][29]
- [Refactor MountingManager into MountingManager + SurfaceMountingManager][37]

iOS Functionalities are included in separate PR #35908
Documentation PR facebook/react-native-website#3010

Next PR [2/2 TextInput accessibilityErrorMessage (VoiceOver, iOS) https://github.com/facebook/react-native/issues/35908](https://github.com/facebook/react-native/pull/35908)
Related facebook/react-native-deprecated-modules#18

## Changelog

[Android] [Added] - Adding TextInput prop accessibilityErrorMessage to announce with TalkBack screenreaders

Pull Request resolved: #33468

Test Plan:
**Android - 20 Jan 2023**
#33468 (comment)

**iOS - 20 Jan 2023**
#33468 (comment)

<details><summary>CLICK TO OPEN OLD VIDEO TEST CASES</summary>
<p>

**PR Branch - Android and iOS 24th June**
[88]: Android - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event (Fabric) ([link][88])

**PR Branch - Android**
[1]. Test Cases of the functionality (Fabric) ([link][1])
[2]. Test Cases of the functionality (Paper) ([link][2])

**Main Branch**
[6]. Android - Runtime Error in main branch when passing value of 1 to TextInput  placeholder prop ([link][6])

**Issues Solved**
[7]. TalkBack error does not clear error on the next typed character when using onChangeText ([link][7])
**Other Tests**
[8]. Setting the TextInput errorMessage state with setTextAndSelection Java API from JavaScript ([link][8])
[9]. Setting the TextInput errorMessage state from fabric TextInput internal state to Java ReactTextUpdate API ([link][9])

</p>
</details>

[1]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Android - Fabric)"
[2]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Android - Paper)"
[3]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (iOS - Fabric)"
[6]: fabOnReact/react-native-notes#12 (comment) "Runtime Error in main branch when passing value of 1 to TextInput  placeholder prop"
[7]: fabOnReact/react-native-notes#12 (comment) "TalkBack error announcement done on next typed character with onChangeText"
[8]: fabOnReact/react-native-notes#12 (comment) "setting the TextInput errorMessage state with setTextAndSelection Java API from JavaScript"
[9]: fabOnReact/react-native-notes#12 (comment) "Setting the TextInput errorMessage state from fabric TextInput internal state to Java ReactTextUpdate API"

[10]: https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setError(java.lang.CharSequence) "AOSP setError"
[11]: https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setError(java.lang.CharSequence) "AccessibilityNodeInfo#setError"
[12]: https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/text/Editable.java#L28-L52 "Editable method replace"
[13]: https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setContentInvalid(boolean) "setContentInvalid"

[20]: 60b6c9b "draft implementation of android_errorMessage "
[21]: 012d92d "add errorMessage to ReactTextUpdate and maybeSetAccessibilityError"
[22]: fabOnReact@cad239b "rename android_errorMessage to errorMessageAndroid"
[23]: fabOnReact@0bae474 "TextInput: keep C++ state in-sync with updated AttributedStrings in Java"
[24]: fabOnReact@0556e86 "TextInput: support modifying TextInputs with multiple Fragments (Cxx side)"
[25]: fabOnReact@7ab5eb4 "AndroidTextInput: support using commands instead of setNativeProps (native change)"
[26]: fabOnReact@b9491b7 "TextInput: support editing completely empty TextInputs"
[27]: fabOnReact@7f1ed68 "Fabric: convertRawProp was extended to accept an optional default value"
[28]: 6e0fa5f "Support optional types for C++ TurboModules"
[29]: fabOnReact@baa66f6 "Announce accessibility state changes happening in the background"

[30]: fabOnReact/react-native-notes#12 (comment) "Change of error state from onChangeText show/hides a TextInput error"
[31]: fabOnReact/react-native-notes#12 (comment) "understanding onChangeText callback"
[32]: #29063 (comment) "Explanation on how TextInput calls SET_TEXT_AND_SELECTION in Java API"
[33]: #33468 (comment) "Explanation of TextInput state management with fabric C++ and JAVA API"
[34]: #33468 (comment) "state updates between fabric and ReactAndroid for the TextInput component"
[35]: https://reactnative.dev/architecture/render-pipeline#react-native-renderer-state-updates "ReactNative renderer state updates"
[35]: fabOnReact/react-native-notes#12 (comment) "Analysis on how AndroidTextInputState.cpp sends updates to ReactTextInputManager"
[36]: #33468 (comment) "discussion on using announceForAccessibility in ReactEditText"
[37]: fabOnReact@29eb632 "Refactor MountingManager into MountingManager + SurfaceMountingManager"
[38]: fabOnReact@733f228 "Diff C++ props for Android consumption"
[39]: fabOnReact@91b3f5d "This diff implement and integrates Mapbuffer into Fabric text measure system"

[40]: #29070 "[Android] Fix letters duplication when using autoCapitalize #29070"

[50]: fabOnReact/react-native-notes#12  "Notes from work on iOS/Android: Text input error for screenreaders #12"
[51]: #30848 "iOS/Android: Text input error for screenreaders #30848"
[52]: #30859 "Android: Error state change (text inputs) #30859"

[61]: eb33c93 "fix annoucement delayed to next character"

[70]: fabOnReact/react-native-notes#12 (comment) "iOS - Paper renderer does not update the accessibilityValue"
[71]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Fabric) after removing changes to .cpp libs"
[72]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Paper) after removing changes to .cpp libs"
[73]: fabOnReact/react-native-notes#12 (comment) "iOS - announcing error onChangeText and screenreader focus"
[74]: fabOnReact/react-native-notes#12 (comment) "iOS - The screenreader announces the TextInput value after the errorMessage is cleared"
[75]: fabOnReact/react-native-notes#12 (comment) "iOS - Exception thrown while executing UI block: - [RCTTextView setOnAccessibiltyAction:]: unrecognized selector sent to instance (Paper) (main branch)"
[76]: #30859 (comment) "iOS - announce lastChar (not entire text) onChangeText and avoid multiple announcements (Fabric)"
[77]: #30859 (comment) "iOS - announces or does not announce the accessibilityError through Button onPress (not onChangeText) (Fabric)"
[78]: #30859 (comment) "iOS - the error is announced with accessibilityInvalid true and does not clear after typing text (onChangeText) (Fabric)"
[79]: #30848 (comment) "iOS - Exception thrown while executing UI block: - RCTUITextView setAccessibilityErrorMessage:]: unrecognized selector sent to instance (iOS - Paper on main branch)"

[80]: fabOnReact@e13b9c6 "RCTTextField was spliited into two classes"
[81]: fabOnReact@ee9697e "Introducing RCTBackedTextInputDelegate"
[82]: fabOnReact@2dd2529 "Add option to hide context menu for TextInput"
[83]: https://github.com/fabriziobertoglio1987/react-native/blob/343eea1e3150cf54d6f7727cd01d13eb7247c7f7/React/Fabric/Mounting/ComponentViews/Text/RCTParagraphComponentAccessibilityProvider.mm#L48-L72 "RCTParagraphComponentAccessibilityProvider accessibilityElements"
[84]: https://github.com/fabriziobertoglio1987/react-native/blob/c8790a114f6f21774c43f0e9b9210e7b35d1c243/React/Fabric/Mounting/ComponentViews/TextInput/RCTTextInputComponentView.mm#L613 "RCTTextInputComponentView method _setAttributedString"
[85]: https://github.com/fabriziobertoglio1987/react-native/blob/c8790a114f6f21774c43f0e9b9210e7b35d1c243/React/Fabric/Mounting/ComponentViews/TextInput/RCTTextInputComponentView.mm#L146 "RCTTextInputComponentView method updateProps"
[86]: https://github.com/fabriziobertoglio1987/react-native/blob/c8790a114f6f21774c43f0e9b9210e7b35d1c243/Libraries/Text/TextInput/RCTBaseTextInputView.m#L150 "RCTBaseTextInputView setAttributedText"
[87]: #30859 (comment) "iOS - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event"
[88]: #30859 (comment) "Android - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event"
[89]: #30859 (comment) "iOS - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event (Fabric)"

[100]: fabOnReact@110b191 "Refactor ViewPropsMapBuffer -> general MapBuffer props mechanism"
[101]: fabOnReact@22b6e1c "In this diff I'm shipping and deleting mapBufferSerialization for Text measurement"

Reviewed By: blavalla

Differential Revision: D38410635

Pulled By: lunaleaps

fbshipit-source-id: cd80e9a1be8f5ca017c979d7907974cf72ca4777
OlimpiaZurek pushed a commit to OlimpiaZurek/react-native that referenced this issue May 22, 2023
…#33468)

Summary:
**Android**: The functionality consists of calling the [AccessibilityNodeInfo#setError][10] and [#setContentInvalid][13] method to display the error message in the TextInput.

**Fixes [https://github.com/facebook/react-native/issues/30848][51] - Adding an accessibilityErrorMessage prop to the TextInput Component**:
**Android**: The prop accessibilityErrorMessage triggers the AccessibilityNodeInfo method [setError][10] which automatically sets the correct properties on the AccessibilityNodeInfo that will inform screen readers of this state. The method calls setContentInvalid(true) and setError(youErrorString) on the AccessibilityNodeInfo.

**Fixes [https://github.com/facebook/react-native/issues/30859][52] -  Detecting changes in the Error state (text inputs)**
**Fabric - Android** - Adding accessibilityErrorMessage to field AndroidTextInputState.
ReactTextInputManager and ReactEditText receive state updates both from [Javascript][32] and [cpp (fabric)][34].
- accessibilityErrorMessage is added to the fabric AndroidTextInputState field
- The updates are received in the ReactAndroid API with method updateState from ReactTextInputManager
- After updating the TextInput text with onChangeText, the update of the accessibilityErrorMessage is triggered with method maybeSetAccessibilityError which triggers [setError][10].

More info:
- An explanation of [state updates between fabric and ReactAndroid for the TextInput component][34]
- [ReactNative renderer state updates][35]

**Paper - Android** - Adding accessibilityErrorMessage to ReactTextInputShadowNode to trigger updates in Paper renderer when accessibilityErrorMessage is changed within the onChange callback.

Related Links (Android):
- [In this diff I'm shipping and deleting mapBufferSerialization for Text measurement][101]
- [This diff implement and integrates Mapbuffer into Fabric text measure system][39]
- [Refactor ViewPropsMapBuffer -> general MapBuffer props mechanism][100]
- [TextInput: support modifying TextInputs with multiple Fragments (Cxx side)][24]
- [TextInput: keep C++ state in-sync with updated AttributedStrings in Java][23]
- [AccessibilityNodeInfo#setError][11]
- [Explanation on how TextInput calls SET_TEXT_AND_SELECTION in Java API][32]
- [Fabric: convertRawProp was extended to accept an optional default value][27]
- [understanding onChangeText callback][31]
- [Editable method replace()][12]
- [Change of error state from onChangeText show/hides a TextInput error][30]
- [AndroidTextInput: support using commands instead of setNativeProps (native change)][25]
- [TextInput: support editing completely empty TextInputs][26]
- [[Android] Fix letters duplication when using autoCapitalize https://github.com/facebook/react-native/issues/29070][40]
- [Support optional types for C++ TurboModules][28]
- [discussion on using announceForAccessibility in ReactEditText][36]
- [ fix annoucement delayed to next character][61]
- [Announce accessibility state changes happening in the background][29]
- [Refactor MountingManager into MountingManager + SurfaceMountingManager][37]

iOS Functionalities are included in separate PR facebook#35908
Documentation PR facebook/react-native-website#3010

Next PR [2/2 TextInput accessibilityErrorMessage (VoiceOver, iOS) https://github.com/facebook/react-native/issues/35908](https://github.com/facebook/react-native/pull/35908)
Related facebook/react-native-deprecated-modules#18

## Changelog

[Android] [Added] - Adding TextInput prop accessibilityErrorMessage to announce with TalkBack screenreaders

Pull Request resolved: facebook#33468

Test Plan:
**Android - 20 Jan 2023**
facebook#33468 (comment)

**iOS - 20 Jan 2023**
facebook#33468 (comment)

<details><summary>CLICK TO OPEN OLD VIDEO TEST CASES</summary>
<p>

**PR Branch - Android and iOS 24th June**
[88]: Android - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event (Fabric) ([link][88])

**PR Branch - Android**
[1]. Test Cases of the functionality (Fabric) ([link][1])
[2]. Test Cases of the functionality (Paper) ([link][2])

**Main Branch**
[6]. Android - Runtime Error in main branch when passing value of 1 to TextInput  placeholder prop ([link][6])

**Issues Solved**
[7]. TalkBack error does not clear error on the next typed character when using onChangeText ([link][7])
**Other Tests**
[8]. Setting the TextInput errorMessage state with setTextAndSelection Java API from JavaScript ([link][8])
[9]. Setting the TextInput errorMessage state from fabric TextInput internal state to Java ReactTextUpdate API ([link][9])

</p>
</details>

[1]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Android - Fabric)"
[2]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Android - Paper)"
[3]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (iOS - Fabric)"
[6]: fabOnReact/react-native-notes#12 (comment) "Runtime Error in main branch when passing value of 1 to TextInput  placeholder prop"
[7]: fabOnReact/react-native-notes#12 (comment) "TalkBack error announcement done on next typed character with onChangeText"
[8]: fabOnReact/react-native-notes#12 (comment) "setting the TextInput errorMessage state with setTextAndSelection Java API from JavaScript"
[9]: fabOnReact/react-native-notes#12 (comment) "Setting the TextInput errorMessage state from fabric TextInput internal state to Java ReactTextUpdate API"

[10]: https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setError(java.lang.CharSequence) "AOSP setError"
[11]: https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setError(java.lang.CharSequence) "AccessibilityNodeInfo#setError"
[12]: https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/text/Editable.java#L28-L52 "Editable method replace"
[13]: https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setContentInvalid(boolean) "setContentInvalid"

[20]: facebook@60b6c9b "draft implementation of android_errorMessage "
[21]: facebook@012d92d "add errorMessage to ReactTextUpdate and maybeSetAccessibilityError"
[22]: fabOnReact@cad239b "rename android_errorMessage to errorMessageAndroid"
[23]: fabOnReact@0bae474 "TextInput: keep C++ state in-sync with updated AttributedStrings in Java"
[24]: fabOnReact@0556e86 "TextInput: support modifying TextInputs with multiple Fragments (Cxx side)"
[25]: fabOnReact@7ab5eb4 "AndroidTextInput: support using commands instead of setNativeProps (native change)"
[26]: fabOnReact@b9491b7 "TextInput: support editing completely empty TextInputs"
[27]: fabOnReact@7f1ed68 "Fabric: convertRawProp was extended to accept an optional default value"
[28]: facebook@6e0fa5f "Support optional types for C++ TurboModules"
[29]: fabOnReact@baa66f6 "Announce accessibility state changes happening in the background"

[30]: fabOnReact/react-native-notes#12 (comment) "Change of error state from onChangeText show/hides a TextInput error"
[31]: fabOnReact/react-native-notes#12 (comment) "understanding onChangeText callback"
[32]: facebook#29063 (comment) "Explanation on how TextInput calls SET_TEXT_AND_SELECTION in Java API"
[33]: facebook#33468 (comment) "Explanation of TextInput state management with fabric C++ and JAVA API"
[34]: facebook#33468 (comment) "state updates between fabric and ReactAndroid for the TextInput component"
[35]: https://reactnative.dev/architecture/render-pipeline#react-native-renderer-state-updates "ReactNative renderer state updates"
[35]: fabOnReact/react-native-notes#12 (comment) "Analysis on how AndroidTextInputState.cpp sends updates to ReactTextInputManager"
[36]: facebook#33468 (comment) "discussion on using announceForAccessibility in ReactEditText"
[37]: fabOnReact@29eb632 "Refactor MountingManager into MountingManager + SurfaceMountingManager"
[38]: fabOnReact@733f228 "Diff C++ props for Android consumption"
[39]: fabOnReact@91b3f5d "This diff implement and integrates Mapbuffer into Fabric text measure system"

[40]: facebook#29070 "[Android] Fix letters duplication when using autoCapitalize facebook#29070"

[50]: fabOnReact/react-native-notes#12  "Notes from work on iOS/Android: Text input error for screenreaders facebook#12"
[51]: facebook#30848 "iOS/Android: Text input error for screenreaders facebook#30848"
[52]: facebook#30859 "Android: Error state change (text inputs) facebook#30859"

[61]: facebook@eb33c93 "fix annoucement delayed to next character"

[70]: fabOnReact/react-native-notes#12 (comment) "iOS - Paper renderer does not update the accessibilityValue"
[71]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Fabric) after removing changes to .cpp libs"
[72]: fabOnReact/react-native-notes#12 (comment) "Test Cases of the functionality (Paper) after removing changes to .cpp libs"
[73]: fabOnReact/react-native-notes#12 (comment) "iOS - announcing error onChangeText and screenreader focus"
[74]: fabOnReact/react-native-notes#12 (comment) "iOS - The screenreader announces the TextInput value after the errorMessage is cleared"
[75]: fabOnReact/react-native-notes#12 (comment) "iOS - Exception thrown while executing UI block: - [RCTTextView setOnAccessibiltyAction:]: unrecognized selector sent to instance (Paper) (main branch)"
[76]: facebook#30859 (comment) "iOS - announce lastChar (not entire text) onChangeText and avoid multiple announcements (Fabric)"
[77]: facebook#30859 (comment) "iOS - announces or does not announce the accessibilityError through Button onPress (not onChangeText) (Fabric)"
[78]: facebook#30859 (comment) "iOS - the error is announced with accessibilityInvalid true and does not clear after typing text (onChangeText) (Fabric)"
[79]: facebook#30848 (comment) "iOS - Exception thrown while executing UI block: - RCTUITextView setAccessibilityErrorMessage:]: unrecognized selector sent to instance (iOS - Paper on main branch)"

[80]: fabOnReact@e13b9c6 "RCTTextField was spliited into two classes"
[81]: fabOnReact@ee9697e "Introducing RCTBackedTextInputDelegate"
[82]: fabOnReact@2dd2529 "Add option to hide context menu for TextInput"
[83]: https://github.com/fabriziobertoglio1987/react-native/blob/343eea1e3150cf54d6f7727cd01d13eb7247c7f7/React/Fabric/Mounting/ComponentViews/Text/RCTParagraphComponentAccessibilityProvider.mm#L48-L72 "RCTParagraphComponentAccessibilityProvider accessibilityElements"
[84]: https://github.com/fabriziobertoglio1987/react-native/blob/c8790a114f6f21774c43f0e9b9210e7b35d1c243/React/Fabric/Mounting/ComponentViews/TextInput/RCTTextInputComponentView.mm#L613 "RCTTextInputComponentView method _setAttributedString"
[85]: https://github.com/fabriziobertoglio1987/react-native/blob/c8790a114f6f21774c43f0e9b9210e7b35d1c243/React/Fabric/Mounting/ComponentViews/TextInput/RCTTextInputComponentView.mm#L146 "RCTTextInputComponentView method updateProps"
[86]: https://github.com/fabriziobertoglio1987/react-native/blob/c8790a114f6f21774c43f0e9b9210e7b35d1c243/Libraries/Text/TextInput/RCTBaseTextInputView.m#L150 "RCTBaseTextInputView setAttributedText"
[87]: facebook#30859 (comment) "iOS - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event"
[88]: facebook#30859 (comment) "Android - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event"
[89]: facebook#30859 (comment) "iOS - accessibilityValue announces correctly with/out errorMessage set with onChangeText or with outside event (Fabric)"

[100]: fabOnReact@110b191 "Refactor ViewPropsMapBuffer -> general MapBuffer props mechanism"
[101]: fabOnReact@22b6e1c "In this diff I'm shipping and deleting mapBufferSerialization for Text measurement"

Reviewed By: blavalla

Differential Revision: D38410635

Pulled By: lunaleaps

fbshipit-source-id: cd80e9a1be8f5ca017c979d7907974cf72ca4777
@fabOnReact fabOnReact unpinned this issue Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant