Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding prop accessibilityUnit to TextAttributesProps (fabric) #3

Merged
merged 14 commits into from
Nov 15, 2022

Conversation

fabOnReact
Copy link
Owner

@fabOnReact fabOnReact commented Nov 11, 2022

Summary

Adds the accessibilityUnit to fabric CPP API (part of PR facebook#35130 Adding support for Android Accessibility TtsSpan API)

The TtsSpan API now supports verbatim, date, and fraction but may not work with other span types like phone numbers, currency, time, measure, and money.

An explanation of the use case is as follows:

  • Time is spelled 1 0 3 0 instead of ten thirty
  • The developer does not have the option to specify if 10m stands for 10 meters or 10 miles, or 10 minutes
  • Phone numbers, for example, 33312344, are not yet supported

Date, time, numbers, and locale lack control of readback details #30849

Implementing the above functionality requires adding a new prop to the react-native API responsible for managing attributes of nested Text.

  • Java and CPP TextAttributes manage the functionality.
  • Nested Text does not correspond to a Widget on Android but an Android TextView Span.

The spans are updated based on a caching mechanism that re-renders the Android TextView only if specific attributes are updated.
The updates are triggered only if the Text or Paragraph Text attributes change.

  • An example of Text attributes is accessibilityRole, font-weight, and backgroundColor.
  • An example of Paragraph attributes is textBreakstrategy. They are updated with different mechanisms because they may change the paragraph layout.
    These mappings are configured in Java, CPP, and Javascript.

As this and PR #33468 require changes to the text attributes, the below work was needed this week:

Related: PR #4 - separating accessibilityUnit from accessibilityMinutes and accessibilityHours

@github-actions
Copy link

github-actions bot commented Nov 11, 2022

Fails
🚫

❗ Base Branch - The base branch for this PR is something other than main or a -stable branch. Are you sure you want to target something other than the main branch?

🚫

📋 Missing Changelog - Can you add a Changelog? To do so, add a "## Changelog" section to your PR description. A changelog entry has the following format: [CATEGORY] [TYPE] - Message.

CATEGORY may be:
  • General
  • iOS
  • Android
  • JavaScript
  • Internal (for changes that do not need to be called out in the release notes)

TYPE may be:

  • Added, for new features.
  • Changed, for changes in existing functionality.
  • Deprecated, for soon-to-be removed features.
  • Removed, for now removed features.
  • Fixed, for any bug fixes.
  • Security, in case of vulnerabilities.

MESSAGE may answer "what and why" on a feature level. Use this to briefly tell React Native users about notable changes.

Warnings
⚠️ 📋 Missing Test Plan - Can you add a Test Plan? To do so, add a "## Test Plan" section to your PR description. A Test Plan lets us know how these changes were tested.
⚠️

Libraries/Components/View/ViewPropTypes.js#L13 - Libraries/Components/View/ViewPropTypes.js line 13 – Requires should be sorted alphabetically (lint/sort-imports)

⚠️

Libraries/Components/View/ViewPropTypes.js#L30 - Libraries/Components/View/ViewPropTypes.js line 30 – 'AccessibilityUnit' is defined but never used. (no-unused-vars)

⚠️

packages/rn-tester/js/examples/Accessibility/AccessibilityExample.js#L1526 - packages/rn-tester/js/examples/Accessibility/AccessibilityExample.js line 1526 – 'pressed' is assigned a value but never used. (no-unused-vars)

⚠️

packages/rn-tester/js/examples/Accessibility/AccessibilityExample.js#L1526 - packages/rn-tester/js/examples/Accessibility/AccessibilityExample.js line 1526 – 'setPressed' is assigned a value but never used. (no-unused-vars)

Generated by 🚫 dangerJS against 39b55e6

SOLUTION: Clearing the cache fixed the issue ([comment](pyg-team/pytorch_geometric#4419 (comment))). The user was experiencing similar issue on arm64.
It reports errors that result from attempts to exceed implementation defined length limits for some object ([link](https://en.cppreference.com/w/cpp/error/length_error)).

"Adding a string prop to TextAttributes.cpp:
Read error message with Android Studio ([screenshot](https://www.icloud.com/iclouddrive/053tE1XWF0rgVUMxmLhUJTJUA#terminating_std_length_error_vector))
Search online for the error message and understand the configuration mistake in the cpp type ([link](https://www.google.com/search?q=terminating+with+uncaught+exception+of+type+std::length_error:+vector))
terminating with uncaught exception of type std::length_error: vector
Verify all changes from [#2](https://github.com/fabriziobertoglio1987/react-native/pull/2/files) are added in [#3](https://github.com/fabriziobertoglio1987/react-native/pull/3/files)
Verify there are no mistakes in the types
Read about APIs that you added to cpp
Read complete logcat errors
Does it build only with the Java Changes?
Commit the java changes and stash cpp
The error std::length_error: vector indicates wrong cpp type
getOrCreateSpannableForText is not called. The method is called on the parent Text, but not on the Span Text.
Review implementation of accessibilityRole in [BaseViewManager](https://github.com/fabriziobertoglio1987/react-native/blob/ec4bc8eb5905f81cebbb4b623acda8245ee5683c/ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManager.java#L247) and [ReactMapBufferPropSetter.kt](http://www.apple.com/)
Probably the method setAccessibilityRole is called directly from kotlin MapBufferPropSetter
Add required changes and debug the functionality
accessibilityRole link is called, but time is not
they have the same cpp configuration, but one of them is not invoked in BaseViewManager. 
This suggest missing configs from the [accessibility link PR](facebook@7b5b114), maybe accessibilityRole Link is triggered with resetAccessibilityDelegate
Find logic in accessibility link PR that resets the Delegate
Before resetting the Delegate, TextLayoutManager adds the Span to the Text. The field is TextAttributes mIsAccessibilityLink. This field is set to true if accessibilityRole = link
mIsAccessibilityLink is set to true in the ReactBaseTextShadowNode #setIsAccessibilityLink sets the value
Debug the value of ReactBaseTextShadowNode #setAccessibilityUnit
Verify if accessibilityUnit is retrieve in that method, because this manager is also for nested text
Check if setIsAccessibiltyLink also is triggered for role time, if not understand why text does not trigger. Yes, it is triggered. We set there the accessibilityRole
- We could follow the same implementation, we add the span based on mIsAccessibilityTtsSpan
- We retrieve get spans of class ReactTtsSpan 
- We follow same implementation from accessibilityLinks
Implement same logic for accessibilityRole time
Verify accessibilityRole time is invoked
Use resetAccessibilityDelegate to add accessibilityUnit info to child Text
Copy missing changes from another prop (for ex. accessibilityRole). setAccessibilityRole is triggered, while setAccessibilityUnit is not triggered.
Review missing diff with [#2](https://github.com/fabriziobertoglio1987/react-native/pull/2/files)
Review the current diff with main branch
Add breakpoint in cpp
SOLUTION: adding the value in [attributedstring/conversion.h](https://github.com/fabriziobertoglio1987/react-native/blob/ec4bc8eb5905f81cebbb4b623acda8245ee5683c/ReactCommon/react/renderer/attributedstring/conversions.h#L1087) will then pass it to [TextAttributesProps as key 24](https://www.icloud.com/iclouddrive/0b8-oKB396zP7Astpyc3puGqQ#ACC_UNIT_passed_as_key_24)
"Complete draft CPP accessibilityUnit settings:
Commit the changes and push
Add more clear log statements and further verify value passed to  accessibilityUnit
Log the value passed from javascript in conversion.h
Add changes from PR [#2](https://github.com/fabriziobertoglio1987/react-native/pull/2/files)
Change logic in conversion.h to add the value passed from javascript"
@fabOnReact fabOnReact marked this pull request as ready for review November 11, 2022 18:58
@fabOnReact fabOnReact marked this pull request as draft November 11, 2022 18:59
Basic working setup to use CPP Map instead of basic string type
Settings copied from accessibilityState (fromRawValue) and from
importantForAccessibility (toString)

the CPP convertRawProps will use two functions to convert the prop:
- fromRawValue which takes as param the prop RawValue
- toString or toBoolean, to*** which convert the RawValue to
  String/Boolean/Number

The error was triggered for missing conversion from RawValue
(AccessibilityUnit field hours) toString.

(std::string) accessibilityUnit.hours would call
toString(accessibilityUnit.hours)

Tasks Details:
Complete draft CPP accessibilityUnit settings:Commit the changes and pushAdd more clear log statements and further verify value passed to  accessibilityUnitLog the value passed from javascript in conversion.hAdd changes from PR #2Change logic in conversion.h to add the value passed from javascript
--
Change the type from String to Hash in PR #3:Plan next tasksFind a prop that uses the Hash type (for ex accessibilityState)Start with a clear git statusCopy the CPP settings from accessibilityState
Parse accessibilityUnit hours value to a string type. The value is already string, but in CPP is not mapped to string.importantForAccessibility uses 2 conversions, fromRawValue and toString. fromRawValue parses the props importantForAccessibility, while toString converts the value from IMPORTANT_FOR_ACCESSIBILITY to string.Follow the same implementation fromRawValue(IMPORTANT_FOR_ACCESSIBILITY), but avoid adding their toString(Important_For_Accessibility) conversion.Pass a static string auto hours = “10”;

"Complete draft CPP accessibilityUnit settings:
Commit the changes and push
Add more clear log statements and further verify value passed to  accessibilityUnit
Log the value passed from javascript in conversion.h
Add changes from PR [#2](https://github.com/fabriziobertoglio1987/react-native/pull/2/files)
Change logic in conversion.h to add the value passed from javascript"
"Change the type from String to Hash in PR [#3](https://github.com/fabriziobertoglio1987/react-native/pull/3/files):
Plan next tasks
Find a prop that uses the Hash type (for ex [accessibilityState](https://github.com/fabriziobertoglio1987/react-native/blob/41173cbeba53b26a3c838d16776daa8d10e57639/ReactCommon/react/renderer/components/view/AccessibilityProps.h#L41))
Start with a clear git status
Copy the CPP settings from accessibilityState"
"Parse accessibilityUnit hours value to a string type. The value is already string, but in CPP is not mapped to string.
[importantForAccessibility](https://github.com/fabriziobertoglio1987/react-native/blob/41173cbeba53b26a3c838d16776daa8d10e57639/ReactCommon/react/renderer/components/view/accessibilityPropsConversions.h#L173-L209) uses 2 conversions, fromRawValue and toString. fromRawValue parses the props importantForAccessibility, while toString converts the value from IMPORTANT_FOR_ACCESSIBILITY to string.
Follow the same implementation [fromRawValue(IMPORTANT_FOR_ACCESSIBILITY)](https://github.com/fabriziobertoglio1987/react-native/blob/41173cbeba53b26a3c838d16776daa8d10e57639/ReactCommon/react/renderer/components/view/accessibilityPropsConversions.h#L193), but avoid adding their toString(Important_For_Accessibility) conversion.
Pass a static string auto hours = “10”;"
[build error](https://www.icloud.com/iclouddrive/083j4A7FWBweG3wFmAnHzDyfQ#call_to_implicitly_deleted_constructor) seems to be caused by the double declaration of primitive AccessibilityUnit:
@fabOnReact fabOnReact changed the title accessibilityUnit CPP settings Adds the accessibilityUnit to fabric CPP API Nov 14, 2022
@fabOnReact fabOnReact changed the title Adds the accessibilityUnit to fabric CPP API Adding prop accessibilityUnit to TextAttributesProps (fabric) Nov 15, 2022
@fabOnReact fabOnReact marked this pull request as ready for review November 15, 2022 15:57
@fabOnReact fabOnReact merged commit 8b23efd into tts-span Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant