Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode #18205

gziolo · 2019-10-31T10:25:54Z

Description

Does anyone know why I can’t run npm run test-e2e -- --puppeteer-interactive to see the browser? I always get a connect ECONNREFUSED 127.0.0.1:<PORT> error. The Chrome browser opens and closes really fast.

This PR resolves the issues by upgrading the puppeteer to the new major version. As usual, new major version produces some warnings during npm install but we can't do anything about it since external packages need to update their package.json entries.

How has this been tested?

npm run test-e2e -- --puppeteer-interactive

I confirmed it works with the latest version of puppeteer.

youknowriad · 2019-10-31T10:29:35Z

We should use the new emulateMedia features https://github.com/GoogleChrome/puppeteer/releases to remove our "disable animations plugiin" and also the environment variable we use to disable animations. Instead we should just put the browser in "prefers-reduced-motion" mode.

gziolo · 2019-10-31T10:33:21Z

We should use the new emulateMedia features https://github.com/GoogleChrome/puppeteer/releases to remove our "disable animations plugiin" and also the environment variable we use to disable animations. Instead we should just put the browser in "prefers-reduced-motion" mode.

Can you file a new issue? I don't want to block this PR as you can't debug code at the moment ...

youknowriad · 2019-10-31T10:34:44Z

Definitely not a blocker, just something I thought of when reading the changelog. I might give it a try after this PR get merged, it should be pretty easy.

gziolo · 2019-10-31T10:35:42Z

Definitely not a blocker, just something I thought of when reading the changelog. I might give it a try after this PR get merged, it should be pretty easy.

You probably also know best how to tackle it. Ping me when you have PR ready, I can review pretty quick 👍

gziolo · 2019-10-31T11:51:26Z

It looks like I upgraded axe and they introduced or updates some rules which cause some tests to fail. I will have to silence them for now, similar to other rules which Gutenberg violates. The rules are:

aria-hidden-focus
aria-input-field-name

gziolo · 2019-10-31T12:07:51Z

It looks like I upgraded axe and they introduced or updates some rules which cause some tests to fail. I will have to silence them for now, similar to other rules which Gutenberg violates. The rules are:

aria-hidden-focus

aria-input-field-name

Done with 8de7f70, added to the list of rules to enable in #15452.

ntwb

👍🏼

afercia · 2019-10-31T13:43:21Z

Thanks @gziolo! Regarding aria-hidden-focus please see my comment on #11631 (comment)

Regarding aria-input-field-name can you please point me at the log or the reference where it happens?

gziolo · 2019-10-31T14:28:15Z

Thanks @gziolo! Regarding aria-hidden-focus please see my comment on #11631 (comment)

Regarding aria-input-field-name can you please point me at the log or the reference where it happens?

Sure thing, the good example is in https://travis-ci.com/WordPress/gutenberg/jobs/251398503#L1066"

FAIL packages/e2e-tests/specs/editor/blocks/table.test.js (19.372s)
  ● Table › displays a form for choosing the row and column count of the table
    expect(received).toPassAxeTests(expected)
    Expected page to pass Axe accessibility tests.
    Violations found:
    Rule: "aria-input-field-name" (ARIA input fields have an accessible name)
    Help: https://dequeuniversity.com/rules/axe/3.4/aria-input-field-name?application=axe-puppeteer
    Affected Nodes:
      tr:nth-child(1) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      178 | 	}
      179 | 
    > 180 | 	await expect( page ).toPassAxeTests( {
          | 	                     ^
      181 | 		// Temporary disabled rules to enable initial integration.
      182 | 		// See: https://github.com/WordPress/gutenberg/pull/15018.
      183 | 		disabledRules: [
      at toPassAxeTests (config/setup-test-framework.js:180:23)
          at runMicrotasks (<anonymous>)
      at Object.<anonymous> (config/setup-test-framework.js:218:2)

afercia · 2019-10-31T14:48:24Z

Ah! Thanks. That seems to relate to the table block: the editable within the table cells needs to be labelled. RIght now, there's nothing there to label the editables, just:
<div role="textbox" ....

Which, in ARIA terms, is equivalent to an input field with no label.

gziolo added [Type] Bug An existing feature does not function as intended [Package] Scripts /packages/scripts labels Oct 31, 2019

gziolo requested review from ajitbohra, nerrad, ntwb and youknowriad as code owners October 31, 2019 10:25

gziolo self-assigned this Oct 31, 2019

Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode

e0ad9db

gziolo force-pushed the fix/puppetteer-interactive branch from efed32d to e0ad9db Compare October 31, 2019 10:27

This was referenced Oct 31, 2019

Components: Modal, add story #18083

Merged

Modal: remove a redundant tab stop on the modal content wrapper #11631

Closed

Testing: Disable new introduced rules in axe core which fail

8de7f70

gziolo requested a review from talldan as a code owner October 31, 2019 12:06

ntwb approved these changes Oct 31, 2019

View reviewed changes

gziolo merged commit ab59ea9 into master Oct 31, 2019

gziolo deleted the fix/puppetteer-interactive branch October 31, 2019 12:34

youknowriad added this to the Gutenberg 6.9 milestone Nov 11, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode #18205

Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode #18205

gziolo commented Oct 31, 2019 •

edited

Loading

youknowriad commented Oct 31, 2019

gziolo commented Oct 31, 2019

youknowriad commented Oct 31, 2019

gziolo commented Oct 31, 2019

gziolo commented Oct 31, 2019

gziolo commented Oct 31, 2019

ntwb left a comment

afercia commented Oct 31, 2019

gziolo commented Oct 31, 2019

afercia commented Oct 31, 2019 •

edited

Loading

Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode #18205

Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode #18205

Conversation

gziolo commented Oct 31, 2019 • edited Loading

Description

How has this been tested?

youknowriad commented Oct 31, 2019

gziolo commented Oct 31, 2019

youknowriad commented Oct 31, 2019

gziolo commented Oct 31, 2019

gziolo commented Oct 31, 2019

gziolo commented Oct 31, 2019

ntwb left a comment

Choose a reason for hiding this comment

afercia commented Oct 31, 2019

gziolo commented Oct 31, 2019

afercia commented Oct 31, 2019 • edited Loading

gziolo commented Oct 31, 2019 •

edited

Loading

afercia commented Oct 31, 2019 •

edited

Loading