Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for navigation in browse mode documents and improve tests for textInfo #16578

Open
Adriani90 opened this issue May 20, 2024 · 1 comment
Labels
audience/a11y-tester Accessibility Testers audience/nvda-dev PR or issue is relevant to NVDA / Add-on developers p5 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.

Comments

@Adriani90
Copy link
Collaborator

Adriani90 commented May 20, 2024

Is your feature request related to a problem? Please describe.

Features related to navigation paterns in Browse mode documents might introduce scripts to move the virtual cursor to specific parts of a document. This usually relies on internal and external textInfo related functions. In such cases, NVDA will use textInfo and will analyse the document character by character to find the desired location where the cursor should be moved to.
However, in many cases textInfo will not be able to provide the correct function so the navigation will fail. This might also be related to external issues where the software does not provide the correct definition of characters, words, paragraphs, boundaries, etc. Also scrolling might fail while navigating by certain paterns.
Also NVDA tries to report the whole element when the virtual cursor lands on it, no matter how much content the element has. However, the purpose of the quick navigation paterns in browse mode is to provide more efficiency, so there should be a limit of how much content of an element needs to be sent to the synthesizer.
Problems identified so far:

Text style navigation

  1. Text Style navigation in MS Word or MS Outlook fails using UIA or the old objectModel while tables and text columns introduce invisible characters and boundaries into the document (Text style navigation: Runtime error, NVDA cannot find desired offset in textInfo when using tables in MS Word with UIA disabled #16459, Navigate to text styles in MS Outlook, NVDA throws key errors and freezes at least for 9 or 10 seconds for every key press #16408 which led to Disable style navigation in Outlook and non-UIA word (duplicate) #16569). See also Text style navigation fails: NVDA throws runtime error and also listindex out of range in MS Word documents with text splitted in multiple sections / columns #16527 (which happens both with UIA enabled and disabled).
  2. Text style navigation fails in Kindle (QT 5 interface) due to unavailable formatting information, this could apply to other interfaces as well, in appmodules where NVDA provides support for browse mode document (see Disable text style navigation in Kindle #16559, which led to Disable vertical, style and text QuickNav commands in Kindle #16571)
  3. Text style navigation is very slow in MS Word or MS Outlook, especially when NVDA has to construct a huge speech uterance to report the whole text region at once when the carret or the virtual cursor lands on it. (see text style navigation: NVDA hangs for at least 1-2.5 seconds when using MS Word with UIA disabled #16458 and Navigate to text styles in MS Outlook, NVDA throws key errors and freezes at least for 9 or 10 seconds for every key press #16408 which also led to Disable style navigation in Outlook and non-UIA word (duplicate) #16569).
  4. Text style navigation is very slow when NVDA has to scan a long complex Microsoft Word document until the end and no next text style has been found. See text style navigation: Severe freeze when using UIA in MS Word and no next or previous style has been found #16546.
  5. Text style navigation throws errors and makes NVDA to be silent when pages in pdf documenta contain only images or when they are empty, see Navigation to next same and different styles on a page with Foto in a pdf document: NVDA throws a list index error and the offset is too big for the buffer, causing NVDA to be silent. #16407 which led to Fix exception when triggering style navigation commands in an empty document #16430.
  6. Text style navigation moves the virtual cursor to unexpected invisible characters such as new line characters, tab characters, space characters etc.

Text paragraph navigation

  1. Text paragraph navigation based on textInfo can cause a crash in NVDA when using QT5 interface like in Kindle, see Text paragraph navigation: NVDA crashes in Kindle #16558 which led to Disable vertical, style and text QuickNav commands in Kindle #16571.
  2. Text paragrpaph navigation based on textInfo can be very sluggish when using UIA in Chromium browsers when no next paragraph was found. See NVDA freezes for at least 45 seconds when no next text paragraph is found while using text paragraph navigation p with UIA enabled in Edge Chromium browser #16436 which led to Fix hanging during text paragraph navigation in UIA edge #16450).
  3. The script for text paragraph navigation based on textInfo moves the virtual cursor to the next or previous text paragraph, and NVDA reports the whole paragraph at once. But when the paragraph contains interactive elements such as links or buttons, NVDA didn't report when these elements start and when they end. See Text paragraph navigation: report when entering and leaving a role while reading the paragraph #16143 which led to Fix text paragraph navigation speech #16145.

Vertical indent paragraph navigation

  1. Vertical indent navigation based on paragraphs defined in textInfo can cause a crash or make NVDA be very sluggish when using QT5 interface like in Kindle, see Text paragraph navigation: NVDA crashes in Kindle #16558 which led to Disable vertical, style and text QuickNav commands in Kindle #16571.
  2. Vertical indent navigation based on paragraphs defined in textInfo can be very sluggish on complex websites, see Vertical navigation: offset too big for buffer causes NVDA to freeze for a while at least for 8 seconds for every key press #16382 which led to Fast vertical QuickNav in Gecko-based textInfos #16385.

Other problems

  1. Wehn in empty browse mode documents of Microsoft Word with the old objectModel, NVDA was throwing errors while trying to find desired textInfo points instead of reporting "no next xx" or "no previous xx", see Vertical navigation: runtime error in empty documents. report "no next aligned paragraph " or "no previous aligned paragraph" instead #16386 which led to Vertical quickNav fix #16388, or Same style and previous style navigation: in an empty MS Word document navigating to next same style and next different style gives an runtime error, navigating to previous styles works properly. #16405 which let to Disable style navigation in Outlook and non-UIA word (duplicate) #16569).
  2. NVDA was throwing exceptions errors when trying to move the virtual cursor or carret to textInfo point when the coordinate of the offset is off screen. See Vertical quick navigation throws errors in some cases #16274 which led to Fix exception during vertical paragraph navigation #16276.
  3. NVDA fails in retrieving the correct offset or textInfo of certain points in some applications probably due to external limitations, see e.g. In Kindle app trying to move to next paragraph via textInfo works incorrectly #16570, textInfo.expand(paragraph) works incorrectly in Chrome. #16530, In Outlook emails, textInfos report inconsistent text in some cases #16448).

Describe the solution you'd like

Manual testing

Develop a manual testing template for browse mode documents including:

  • Consider creating a test group for your feature via the groups.io platform and promote that group among the NVDA community communication channels
  • Test whether the script and its key gesture is working without errors in the main browse mode documents such as Microsoft Word, Microsoft Excel, Microsoft Outlook, Mozilla Firefox, Google Chrome, Microsoft Edge, Opera, Brave, Kindle, Steam, Microsoft Teams, Adobe reader etc.
  • Test the script and its key gesture in Microsoft Office and Chromium browsers both (I) when using UIA and (II) when UIA is disabled or set to "only when necessary".
  • Test whether NVDA is sluggish or crashes while navigating to the desired part of the main browse mode document with the quick navigation key gesture in complex documents / websites (consider if NVDA really needs to read the whole content of the element when the virtual cursor lands on it, e.g. in case of a long text paragraph or a long text region where the text is written in the same style. Maybe it makes sense to let NVDA report only the first 500 characters. to improve performance)
  • Test whether NVDA reports properly when it didn't find next or previous element also in empty documents or in browse mode documents which contain only images
  • Test whether NVDA reports the start and the end of an interactive element when navigating with the quick navigation key gesture (e.g. "link x, end of link", even when the link is part of a non interactive region such as text sentence, paragraph etc.)
  • Test whether NVDA moves the virtual cursor in browse mode at the first visible character of the element to which the script with the key gesture should navigate (e.g. to the first letter of a sentence, of a paragraph, of a link etc.)
  • Test whether NVDA throws errors / exceptions when the element which the script + the quick navigation key gesture should navigate to, is not visible on the screen (off screen element)
  • Test whether NVDA reports "not supported in this document" when the navigation patern does not make sense or should not be supported for certainn browse mode documents (e.g. Microsoft Excel or other application where the navigation patern might not work at all).
  • Test whether retrieving textInfo points works properly by using coresponding commands in the NVDA Python console, and consider reporting failures to the developers of the application which fails. Meanwhile, consider disabling the script completely for the application if the navigation does not work. For some examples, See see issues In Kindle app trying to move to next paragraph via textInfo works incorrectly #16570, textInfo.expand(paragraph) works incorrectly in Chrome. #16530, In Outlook emails, textInfos report inconsistent text in some cases #16448.

Describe alternatives you've considered

Automated testing

Develop a robot to perform all steps like in the referenced issues above, maybe the provided test case documents could be used for the robot to perform the tests.

Additional context

Adding new features to the browse mode of NVDA can be tricky and very complex due to the various level of browse mode support in different applications. In order to avoid unexpected issues, such features need extensive testing before being accepted into the core.

@Adriani90
Copy link
Collaborator Author

cc: @seanbudd, @gerald-hartig

@seanbudd seanbudd added p5 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority audience/nvda-dev PR or issue is relevant to NVDA / Add-on developers audience/a11y-tester Accessibility Testers triaged Has been triaged, issue is waiting for implementation. labels May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
audience/a11y-tester Accessibility Testers audience/nvda-dev PR or issue is relevant to NVDA / Add-on developers p5 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.
Projects
None yet
Development

No branches or pull requests

2 participants