test: add a11y reporting using axe-puppeteer

[stephenmathieson]

Happy WordCamp contributor day!

Description

This patch introduces accessibility testing using axe-puppeteer. Each one of the existing E2E tests now runs AxePuppeteer#analyze() on the relevant portion(s) of the DOM and outputs the returned a11y violations to stderr.

In time, as these accessibility problems are fixed, the logging could be replaced with assertions, ensuring no "new" problems are introduced into the codebase.

This patch is similar to #10695, but uses a more complete a11y testing tool and does not introduce new test failures.

How has this been tested?

Running npm run test-e2e locally.

Screenshots

Types of changes

New testing features/utilities.

Checklist:

• My code is tested.
• My code follows the WordPress code style.
• My code follows the accessibility standards.
• My code has proper inline documentation.

[aduth]

Each one of the existing E2E tests now runs AxePuppeteer#analyze() on the relevant portion(s) of the DOM and outputs the returned a11y violations to stderr.

Would it not be reasonable to run the analysis on the entire document after every test case?

Purely from the perspective of the developer experience of authoring tests, I'd neither want this to be possible to overlook, nor a hurdle in making the decision on what should be tested.

[stephenmathieson]

Hey Andrew,

Yes, it would absolutely be possible. We opted to only analyze the relevant portions of the DOM for a two major reasons:

1. reduce duplicate issue count
2. reduce the time spent doing analysis

If neither of these were a concern, we would have structured this differently. Rather than editing every test file, we'd just tap into the global afterEach hook and do something like:

afterEach(async () => {
  const axe = new AxePuppeteer(page)
  const results = await axe.analyze()
  logA11yResults(results)
})

This would have been much simpler, and would have found most of the relevant issues, but would have slowed down the test suite by more than a little and would produce a significant number of duplicate issues.

[gziolo]

This was superseded by #13241, we still need to integrate new matcher with e2e test suite but this is going to be integrated in the way @aduth suggested:

Would it not be reasonable to run the analysis on the entire document after every test case?

Purely from the perspective of the developer experience of authoring tests, I'd neither want this to be possible to overlook, nor a hurdle in making the decision on what should be tested.

Thanks for all your excellent work done here which helped us to make it solid. Much appreciated. Your involvement a WC US and upfront preparation with axe-puppeteer was amazing ❤️