Detect caret movement using events, except for IA2Web #9933

LeonarddeR · 2019-07-16T06:34:23Z

Link to issue number:

Fixes #9928
Follow up of #9773

Summary of the issue:

In text fields, when moving the caret with caret movement commands, such as the arrows, NVDA relies on bookmarks to find out whether the caret has moved. In short:

You press left arrow
NVDA creates a bookmark of the current caret position
NVDA executes left arrow
For up to the caret movement timeout, NVDA tries to find out whether the caret has moved by creating new bookmarks and comparing them against the old.

However, for UIA, this comparison fails, as creating a bookmark at the end of a range and then removing one character from the end of the range results in a new bookmark that is equal to the former.

Description of how this pull request fixes the issue:

This pr introduces a different approach based on code by @codeofdusk in #9773. In first instance, this code caused #9786 to occur (i.e. editors in Chrome reporting the wrong caret position).

Compared to #9773, flow is now as follows:

In a loop in EditableText._hasCaretMoved, NVDA processes pedning events
If there is a focus event pending, the loop is exited
NVDA tries to find out whether the caret position has changed by creating a textInfo at the caret position
new code: only when this succeeds, NVDA checks for pending caret and textChange events. If the object that contains the caret has caretMovementDetectionUsesEvents set to False, it still ignores the caret events. This is what we do in IA2Web objects.

Testing performed:

Tested that deleted characters/words are again reported in UIA controls, such as MS Word
Tested Notepad, Wordpad, Word Without UIA, Thunderbird, LibreOffice, start menu edit field, legacy command consoles and Skype Electron. Made sure that the caret was still correctly reported in all of them when moving with arrow keys or deleting.

Known issues with pull request:

We can't set the caretMovementDetectionUsesEvents attribute to True on editableText.EditableText and then override it in other places, such as IA2Web. This is because EditableText precedes the IA2Web class as well as other classes in the mro Setting it on EditableText directly therefore makes it impossible for other classes to override the attribute. I solved this by making it a magic property on EditableText that first tries to call super before returning the default.

Change log entry:

None

codeofdusk · 2019-07-16T19:07:14Z

Re name, I think useCaretEvents is probably descriptive enough if documented.
In the Windows 10 start menu search field, backspacing characters no longer seems to announce anything.
In UIA consoles, backspacing quickly causes the last character of the prompt (i.e. ">" for cmd) to be occasionally reported.

LeonarddeR · 2019-07-16T19:37:07Z

Caret events are also used for selection if applicable. May be something like caretMovementUsesCaretEvents or caretMovementUsesEvents. Backspacing in the start menu has always been problematic for me, particularly because of the suggestions appearing. I will r"look into it though.

LeonarddeR · 2019-07-17T04:12:11Z

Apart from the start menu issue, all seems fixed now. I need to investigate that on Master builds though, I'm pretty sure it has never worked very well.

… consoles

LeonarddeR · 2019-07-17T05:01:55Z

The code now also listens for textChange events, as they help in the case where text is removed. This fixes the start menu issue for me.

LeonarddeR · 2019-07-17T05:09:50Z

@feerrenrut: I think it is pretty hard to make sure we have everything right in this pr. THerefore, I propose reviewing and merging it as soon as possible, then patching it afterwards for cases that have problems with this new behavior.

codeofdusk · 2019-07-17T05:16:22Z

For a change log entry, how about something like:

== Bug Fixes ==

Improved reporting of deleted text for some controls. (Detect caret movement using events, except for IA2Web #9933)

codeofdusk · 2019-07-17T05:21:33Z

UIA consoles are indeed fixed. Start menu says "blank" once when backspacing the last character, then nothing. Ideally it would function like the Windows run dialog and console (i.e. say nothing when all characters have been deleted) but this is still an improvement.

LeonarddeR · 2019-07-17T05:22:15Z

I don't think anything has improved. It regressed with the use of python 3 and had to be implemented differently. Unless you know a particular improvement I missed?

codeofdusk · 2019-07-17T05:26:35Z

I don't think anything has improved. It regressed with the use of python 3 and had to be implemented differently. Unless you know a particular improvement I missed?

Current master says "blank" repeatedly when deleting characters in the Windows 10 start search field. This new approach only says it once, and maybe could be made to say it not at all?

LeonarddeR · 2019-07-17T05:34:07Z

Hmm that improves things indeed. The fact that this fails in master makes exactly clear why using bookmarks with UIA is not the reliable way to go. I'm afraid that NVDA receives a textChange event when pressing backspace the first time. after the field is blank. I will look at this briefly.

codeofdusk · 2019-07-17T05:35:53Z

Hmm that improves things indeed. The fact that this fails in master makes exactly clear why using bookmarks with UIA is not the reliable way to go. I'm afraid that NVDA receives a textChange event when pressing backspace the first time. after the field is blank. I will look at this briefly.

Once it's fixed, maybe a similar approach can be used for UIA consoles as well? (so we don't have to disable the new approach)

LeonarddeR · 2019-07-17T05:39:42Z

I think for this to work in UIA consoles, we'd have to distinguish between caret and textChange events.

LeonarddeR · 2019-07-17T06:03:06Z

The problem in consoles where the last character of the prompt is reported can also be seen in legacy consoles. This really has to do with events coming in pretty late, i.e. pressing backspace results in an event that is received by NVDA around half a second later.

LeonarddeR · 2019-07-17T06:23:13Z

After the most recent commit, I could no longer reproduce the issue in consoles and therefore I restored the behavior for UIA consoles. In the start menu search field, it still can be observed, though you'd have to remove text pretty quicly. I don't think there's anything we can use to improve this, as said, events are coming pretty late.

feerrenrut

Thanks @LeonarddeR and @codeofdusk. Overall this looks good, just pending some clarification about the timeout comment.

feerrenrut · 2019-07-24T15:38:10Z

source/editableText.py

+			else:
+				# Caret events are unreliable in some controls.
+				# Only use them if we consider them safe to rely on for a particular control,
+				# and only if they arrive within 20 mili seconds after causing the event to occur.


This comment says 20 ms but useEvents_maxTimeoutMs is 10

Ah, I'm sorry. I first hard coded the 20 ms, but then decided to put it into a constant. Just forgot to change it :)

codeofdusk · 2019-07-27T15:32:28Z

As of latest master, I can still reproduce the issues with UIA consoles, although they occur much less frequently.

LeonarddeR · 2019-07-27T17:36:08Z

How about increasing the _useEvents_maxTimeoutMs timeout in editabletext?

codeofdusk · 2019-07-28T04:05:35Z

How about increasing the _useEvents_maxTimeoutMs timeout in editabletext?

Tried 20, 30, and 40 ms with no success.

A regression caused during Python 3 migration. NVDA was announcing the final character of the prompt after backspacing text quickly. This can result in NVDA announcing "greater" after backspacing a line of text. Disable caret events in the UIA console and legacy consoles. This PR disables the caret movement event detection from #9933

In Mintty, the last character of the prompt was being read when quickly deleting text, just as in Windows Console (see this comment and the "summary of the issue" section in #9986). This change disables caret events for all Terminal objects by overriding NVDAObjects.behaviors.Terminal._get_caretMovementDetectionUsesEvents, instead of only Windows Console like in #9986. Closes #1348. Related to #9933, #9986.

Detect caret movement using events, with IA2Web as an exception

3deec04

LeonarddeR requested a review from feerrenrut July 16, 2019 06:34

Revert unneeded change to the NVDAObjects.UIA package

d811a42

Leonard de Ruijter added 2 commits July 17, 2019 06:52

Rename property, make implementation clearer on editableText, fix UIA…

07a2791

… consoles

Also listen for textChange events

d99a31b

Only listen for events during 10ms after invoking the gesture

1385ac2

LeonarddeR mentioned this pull request Jul 19, 2019

Speak typed words based on TextInfo if possible #8110

Closed

feerrenrut suggested changes Jul 24, 2019

View reviewed changes

Review action

0541f7f

LeonarddeR requested a review from feerrenrut July 24, 2019 17:24

feerrenrut approved these changes Jul 24, 2019

View reviewed changes

feerrenrut merged commit 75c8d00 into nvaccess:threshold Jul 24, 2019

nvaccessAuto added this to the 2019.3 milestone Jul 24, 2019

josephsl mentioned this pull request Jul 24, 2019

What's new and readme: we are moving to Python 3.7 #9942

Merged

feerrenrut mentioned this pull request Jul 24, 2019

Python 3: when in UIA based edit field, the deleted character is not spoken when pressing backspace #9928

Closed

This was referenced Jul 28, 2019

Disable caret events in the UIA console #9985

Closed

V2 of disable caret events in the UIA console #9986

Merged

This was referenced Aug 13, 2019

NVDA doesn't announce typed characters in putty #1348

Closed

Disable caret events in all terminal programs #10118

Merged

LeonarddeR added the BabbageWork Pull requests filed on behalf of Babbage B.V. label Oct 11, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Detect caret movement using events, except for IA2Web #9933

Detect caret movement using events, except for IA2Web #9933

LeonarddeR commented Jul 16, 2019 •

edited

Loading

codeofdusk commented Jul 16, 2019

LeonarddeR commented Jul 16, 2019 via email

LeonarddeR commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019

codeofdusk commented Jul 17, 2019

codeofdusk commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 via email

codeofdusk commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 via email

codeofdusk commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 via email

LeonarddeR commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 •

edited

Loading

feerrenrut left a comment

feerrenrut Jul 24, 2019

LeonarddeR Jul 24, 2019

codeofdusk commented Jul 27, 2019

LeonarddeR commented Jul 27, 2019

codeofdusk commented Jul 28, 2019

Detect caret movement using events, except for IA2Web #9933

Detect caret movement using events, except for IA2Web #9933

Conversation

LeonarddeR commented Jul 16, 2019 • edited Loading

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing performed:

Known issues with pull request:

Change log entry:

codeofdusk commented Jul 16, 2019

LeonarddeR commented Jul 16, 2019 via email

LeonarddeR commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019

codeofdusk commented Jul 17, 2019

codeofdusk commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 via email

codeofdusk commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 via email

codeofdusk commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 via email

LeonarddeR commented Jul 17, 2019

LeonarddeR commented Jul 17, 2019 • edited Loading

feerrenrut left a comment

Choose a reason for hiding this comment

feerrenrut Jul 24, 2019

Choose a reason for hiding this comment

LeonarddeR Jul 24, 2019

Choose a reason for hiding this comment

codeofdusk commented Jul 27, 2019

LeonarddeR commented Jul 27, 2019

codeofdusk commented Jul 28, 2019

LeonarddeR commented Jul 16, 2019 •

edited

Loading

LeonarddeR commented Jul 17, 2019 •

edited

Loading