Consistent accessibility tree outputs

[jelbourn]

Description

Browsers should investigate and work towards generating consistent accessibility tries from the same DOM + CSS.

Rationale

All browsers produce subtly different accessibility trees for the same DOM + CSS. Some examples:

• CSS display property impacts the a11y tree inconsistently

• https://www.powermapper.com/tests/screen-readers/aria/

• https://a11ysupport.io/

• For something more hands-on, on a macOS device, using VoiceOver to interact with this W3C datepicker example gives slightly different results between Chrome, Safari, and Firefox:

  Chrome:
  Date, dialog pop up, Menu pop-up combobox
  (date format: mm/dd/yyyy)
  Choose date, dialog with 8 items, Entering October 2022 table. 12, October 2022, table, 7 columns, 7 rows

  Safari:
  Date, dialog pop up, Menu pop-up combobox
  (date format: mm/dd/yyyy)
  Choose date, web dialog, with 8 items 12, Entering October 2022. 12, October 2022 table, 7 columns, 7 rows

  Firefox:
  Date, combobox
  (date format: mm/dd/yyyy)
  Choose date, web dialog, with 8 items 12, Entering October 2022. 12, October 2022 table, 7 columns, 7 rows

While some differences are mostly innocuous ("dialog" vs "web dialog"), having any sort of standardization or interop testing would be a huge boon to all web developers trying to build accessible experiences.

Investigation Roadmap

See #211 (comment)

(Description updated by @foolip Oct 31.)

[karlcow]

do you mean the vocabulary should be standardized?

[jelbourn]

do you mean the vocabulary should be standardized?

Not exactly. I'm not an expert here, so it's possible that I'm missing something in my explanation.

My understanding of the world today is that:

• Each browser produces an accessibility tree from a page's DOM + CSS
• Assistive technology, such as screen readers, operate from this accessibility tree
• There's no standard that describes how browsers produce or structure this accessibility tree
• The easiest way to observe these differences is use the same screen reader on different browsers (Chrome, Safari, and Firefox do all expose some debug information about the a11y tree, but in such different formats that direct comparison isn't really possible)

I see value in investigating these differences between browsers and potentially setting up tests that keep things consistent across platforms.

[chrishtr]

• Assistive technology, such as screen readers, operate from this accessibility tree

They use the tree in an indirect way (see below).

• There's no standard that describes how browsers produce or structure this accessibility tree

The "universal" accessibility tree is not in a spec, but the mapping to OS APIs is (see here).

I see value in having some investigating these differences between browsers and potentially setting up tests that keep things consistent across platforms.

I agree. Unfortunately, there is not yet WPT infrastructure to test the OS mappings, or even to test the computed ARIA roles of elements. (And there is--by design--no web developer exposed API for any of these things.) I think we should fix this infrastructure problem in 2023 so that we can test interop for the accessibility tree in Interop 2024.

[karlcow]

The "universal" accessibility tree is not in a spec

Probably this should be discussed with W3C accessibility WGs.
Not sure Interop can do anything about it. In addition to what @chrishtr mentioned wrt the testing capabilities.

[foolip]

@zcorpan I know that Bocoup has done a lot of work in this space. Do or colleagues have thoughts on the test infra side of this?

[chrishtr]

I think we should treat this issue as a proposal for an Investigation area around improved accessibility testing in WPT. I'm gathering additional feedback now from Chrome a11y team experts and will make a more precise proposal soon.

[chrishtr]

I've finished gathering support from within Chrome a11y experts. I'd like to update this proposal to the following:

Proposal: Accessibility Interop Testing
Type: Investigation effort
Goal: enable interoperability testing of browser accessibility features in WPT. Interoperable means specified behaviors such as in core accessibility mappings, html accessibility mappings, and aria computed roles. None of these are currently testable via WPT. A secondary goal would be to investigate testing of interactive accessibility actions to and from OS APIs (and whether that is achievable without further specifications).

How: form an investigation group that works towards an accepted wpt.fyi RFC for accessibility testing, and builds prototypes in browsers to verify this RFC is workable. It would receive a 100% score for all browsers if the RFC was accepted and at least one complete prototype implementation exists for at least one browser that runs some accessibility tests on wpt.fyi. This group will meet monthly through 2023 to work towards this goal.

---

Answers to the remaining questions from the RFC text for an investigation effort:

• Is this area a problem for web developers? Is this area a problem for users?

Developers very often experience lack of interop with a11y, which is always the result for any feature unless there is at least some interop testing in place. See also the links in the original comment for this issue. Users suffer as a result.

• How widely used is this feature?

A large fraction of the world's population have one need or another for a11y features (see here for example). This page indicates up to 20% of website visitors could be affected.

• Specification for this feature, if any. See links above under Goal.

[zcorpan]

@zcorpan I know that Bocoup has done a lot of work in this space. Do or colleagues have thoughts on the test infra side of this?

We've been working on ARIA AT which is end-to-end screen reader tests, i.e. a test is some HTML+CSS+JS (currently the tests are based on APG design pattern examples but could be any tests) with a particular user interaction (pressing keys usually) and the assertions are done on the spoken output. This setup has no insight at all into what the browser's accessibility tree is like, or what is exposed to the OS accessibility API. But interop bugs somewhere in the "stack" can still be caught if they affect the spoken output. See https://github.com/w3c/aria-at-automation for work to support automation of these tests.

Maybe AOM or WebDriver accessibility primitives (w3c/webdriver#1439 and w3c/webdriver#1441) could help to test the browser's accessibility tree.

For testing OS APIs, I believe there are manual tests in wpt (e.g. /accname/), and IIRC there's tooling somewhere that automates those tests (not baked into wpt infra).

[cookiecrook]

Testing the AccName spec (using computedLabel) may be a good first step:

w3c/accname#174

[cookiecrook]

Interop testing of some TBD accessibility tree was one of the long-term original goals of AOM. One major complication is that WebKit based its internal tree off the RenderTree, compared to Gecko, which starts off the DOM tree. Chromium made a major implementation change a few years ago to more closely align with Gecko, but all 3 are still fairly different.

All of the engine-internal accessibility trees mentioned here are distinct from the platform accessibility API trees, so each engine has other mapping outputs from its internals to the platform trees. These are what’s referenced by the AAM specs: Core-AAM, HTML-AAM, etc.

IIRC where the AOM and ARIA groups landed for interop in the shorter term was:

1. Focus on the automation properties that should be interoperable regardless of implementation. WebDriver computedLabel and computedRole are the first. Add more as needed.
2. Use those to test the more complicated parts of accessibility interoperability, like the accessible name computation algorithm. (This testing recently became possible due to the WebDriver updates. Linked in prior comment.)
3. Add and test more accessibility automation properties as needed and agreed. For example, there is no standard way to verify that a checkbox is checked, regardless of whether it’s an HTML checkbox input or an ARIA checkbox/switch/etc.
4. If/When the interop testing gets around to computedParent/computedChild in the accessibility automation contexts, then we could have a comparison of where the browser internal trees differ and whether it’s necessary to start discussing the possibility of a standardized web accessibility tree, likely separate from the rendering engine internal accessibility tree, and the platform-specific accessibility trees.

As specified elsewhere, WebKit’s potential acceptance any investigations for WPT would be conditional on a scoring change such that investigations should not penalize the score of any implementation.

[cookiecrook]

Cc @spectranaut @alice @cyns @jnurthen

[foolip]

Thank you for proposing consistent accessibility tree outputs for inclusion in Interop 2023.

We are pleased to let you know that this proposal was accepted as part of the Accessibility Testing investigation effort. You can follow the progress of this investigation on the Interop 2023 dashboard.

For an overview of our process, see the proposal selection summary. Thank you for contributing to Interop 2023!

[cookiecrook]

@foolip wrote:

Thank you [@jelbourn] for proposing consistent accessibility tree outputs for inclusion in Interop 2023.
We are pleased to let you know that this proposal was accepted

For what it's worth, I'm all for an InterOp 2023 Accessibility Investigation, but "consistent accessibility tree outputs" is likely to be unachievable in 2023, for the reasons I mentioned above.

I'd love to help with the scoping of this investigation, and with some of the effort.

[cookiecrook]

Actually I see now @chrishtr proposed a reduced scope (unrelated to tree consistency) that several of us at Apple think is much more achievable.

Goal: enable interoperability testing of browser accessibility features in WPT. Interoperable means specified behaviors such as in core accessibility mappings, html accessibility mappings, and aria computed roles. None of these are currently testable via WPT. A secondary goal would be to investigate testing of interactive accessibility actions to and from OS APIs (and whether that is achievable without further specifications).

I believe all or most HTML-AAM and role mapping tests are achievable using get_computed_role, while the AccName tests I'm starting on are mostly achievable using get_computed_label. There may be some minor differences related to pseudo-elements, UA Shadow DOM components, etc, but we can address these when we come to them.

I am not yet certain how we'd test the CoreAAM mappings as proposed, since those are all platform API specific, and would need platform-level verification that is currently outside the scope of WPT, if I understand correctly. @zcorpan may have some ideas for how to incorporate those.

[cookiecrook]

SVG-AAM and DPUB-AAM may be similarly low hanging fruit that could be part of this investigation.

[cookiecrook]

I'd like to move the /accname tests into /accessibility/manual, so that I can start on a set of automated tests covering AccName in /accessibility/accname. See web-platform-tests/wpt#38502

[gsnedders]

I am not yet certain how we'd test the CoreAAM mappings as proposed, since those are all platform API specific, and would need platform-level verification that is currently outside the scope of WPT, if I understand correctly. @zcorpan may have some ideas for how to incorporate those.

Further, as I understand it, it's unclear how many bugs testing Core-AAM would actually find, and it's by far the hardest thing to do from an infrastructure point of view. It seems like the investigation project should be much more heavily skewed towards the ARIA roles/labels the browser believes things have.

[zcorpan]

We'd need some way of interfacing with the platform accessibility API to test core-aam. Or maybe the browser "knows" what the mapping is, and can expose it to WebDriver?

Maybe testing core-aam is more of the realm of ARIA-AT than wpt, although ARIA-AT doesn't currently directly test the platform accessibility API either. cc @jugglinmike