Regression in reading math 2022.1

[NSoiffer]

Steps to reproduce:

1. Get the MathCAT addon. Alternatively download MathPlayer or use the Acess8Math addon.
2. Download the trial version of MathType (I think this is needed, but I'm not 100% certain).
3. Open the file: nvda-math-bug.docx
4. Navigate to the math in the first line. You will hear nothing or "graphic ..." This is math written by MathType. You should hear something like "the square root of a squared plus b squared end root".
5. Navigate to the math on the second line. This will read properly. The was written by Word's built-in math editor

Versions and Settings

In 2021.3.5, the first line will read properly, but not the second (the second will read x2 plus y2). If you go to settings, advanced, and turn on "Use UI Automation to access Microsoft Word document controls when available", the situation will be similar to what happens in 2022.1.

In 2022.1, if you use this control and set the value to anything besides "always", you will hear the MathType equation but not the Word one. This is true even if you set it to the already set "default" setting. Restarting NVDA reverts to the initial behavior. NVDA is acting like it is not using this setting until the advanced settings dialog is used and then forgetting that on restart. This is true even if "only when necessary" is checked. Simply selecting and then unselecting the "I understand that changing these settings..." causes the settings to be used (i.e., the "I understand" box is left unselected).

In 2022.2beta2, NVDA seems to be back to the 2021.3.5 behavior, so probably fixed.

Suggestion

What would be MUCH, MUCH better is for NVDA to work on both types of math without having to go to the NVDA advanced settings. If NVDA is moving towards using UIA, would it be possible to add a special check for the MathType object and switch to using MSAA to get at it? Or if that isn't the direction NVDA is heading, would it be possible to use UIA if a Word math object is detected?

[seanbudd]

From the 2022.1 change log:

- In Microsoft Word accessed via UI Automation, NVDA will now make use of mathPlayer to read and navigate Office math equations. (#12946)
  - For this to work, you must be running Microsoft Word 365/2016 build 14326 or later. 
  - MathType equations must also be manually converted to Office Math by selecting each, opening the context menu, choosing Equation options, Convert to Office Math.

Could you confirm the version of Word you are using?

I don't believe we have intentionally fixed this issue for 2022.2beta2, and I am surprised the behaviour has changed.

[NSoiffer]

2022.1 did add the ability to read and navigate Office math equations. It works with many versions of Word including my older (desktop) version of Word 2016 (with the appropriate UIA settings). The change log is being pretty conservative in terms of the versions it works with. It was a really great addition.

However, asking someone to change the editor they use for math is a big ask. Also, the conversion changes the document, so it is at best a work around, not a proper solution. You wouldn't claim that NVDA reads Word documents if it required them to export it to an HTML file and read that, but that is the same effect for math.

The "ask" I have is to support both forms of math. If the UIA settings are one way, you get office math; set the other way, you get MathType math. Isn't there a way so that in UIA "mode", you can spot the MathType object, temporarily change the UIA settings, get the MathML, and then revert the settings? That would avoid requiring users to jump through hoops (either changing the document or changing the NVDA advanced settings) to read the kind of math in the document they are reading.

[michaelDCurran]

Although I totally agree that this is an issue, there is no current way for us to directly map from a UIA document position to an Office object model position. So switching from uIA to Office object model at runtime in the middle of a document is not really possible. The only case we could support is where the caret is directly on the equation, as then we could see where the caret is in the other mode.
In short this is actually very complex to address, but we will get to this when we can.

[NSoiffer]

Thanks for explaining why it is hard. At the risk of asking the obvious: what about the other way around? If you are using the Office object model and come to a word equation object (I think it has some different id than other objects), then is it possible to use UIA at that point?

[michaelDCurran]

It is no better the other way around either sorry to say. But as I mentioned, we could at very least support it correctly at the caret position.

[NSoiffer]

Closing this because #17554 is a more uptodate repeat of this problem.

[CyrilleB79]

@NSoiffer why did you open #17554 while there was already this issue?

Please note that the current issue has been triaged and prioritized while is not (yet).

[NSoiffer]

@CyrilleB79: I missed the old issue when I did a quick search before entering the new issue.

I was "provoked" to write up the issue (apparently again) after getting a few more "math doesn't read in Word" emails over the last month (and in a meeting or two) and having to ask if they were using MathType or Word and what their setting was for using UIA in Word. Because the Word math editor is becoming accessible for someone who is blind, I think more people are using it these days than used to use it. But MathType has some advantages over the Word editor, so many people still use that. That creates this problem where people have to know which editor created the math, be educated enough about this somewhat technical difference in using MSAA and UIA, and know how to switch back and forth depending on the document they are reading.

@michaelDCurran said this is only able to be fixed if the math is at the carrot position. That's the case when it says "graphic MathType 6 object". In other cases, the math is just skipped/silent. I presume this is because querying text runs that contains objects in Word via UIA returns empty strings for the objects. Handling this one "at the cursor" case might be a good start, but it also might lead to more confusion in that math would sometimes read correctly and sometimes wouldn't read at all.

Here's a possible solution: When a Word document is read by NVDA, have NVDA or maybe MathCAT (which will get built into NVDA) look at the UIA settings and at the math in the document and see if the settings are appropriate for the math. If not, I can see two things that might be done either automatically or with a dialog letting the user decide:

1. (Easier) -- change the UIA settings. Potentially a cache could be added so that if someone is looking at multiple documents in a session, the settings are switched automatically after the first time (assuming a dialog asked whether they should be changed) when the focus shifts back to the Word document.
2. (Harder) -- rewrite the math from one form to another. MathType has the ability to change Word math into MathType math. This might require the paid version of MathType. I don't know of a way to get Word math to become MathType math, but I suspect one could write some Visual Basic code that exports the MathType math as MathML and then calls on Word to import it into it into the and document to replace that math. Changing the document contents seems like a bad idea to me.

The first approach fails if the document has both forms of math. This is likely very rare but as the test document shows, is possible.

Thoughts?