cy.task('acceptMetamaskAccess') does not work in Codespaces

[fionnachan]

Thanks for making synpress!

The same test works fine on my local machine (Windows) but failed within Codespaces.

it("Should connect wallet using MetaMask successfully", () => {
  cy.visit(`/`);
  cy.findByText("Agree to terms").should("be.visible").click();
  cy.findByText("MetaMask").should("be.visible");
  cy.findByText("Connect to your MetaMask Wallet").click();
  cy.acceptMetamaskAccess().then(connected => {
    expect(connected).to.be.true;
    cy.switchToCypressWindow().should("be.true");
  });
});

  1 passing (48s)
  1 failing

  1) Login Account
       Should connect wallet using MetaMask successfully:
     CypressError: `cy.task('acceptMetamaskAccess')` failed with the following error:

> page.waitForNavigation: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for navigation until "load"
============================================================

https://on.cypress.io/api/task
      at http://localhost:3000/__cypress/runner/cypress_runner.js:145655:78
      at tryCatcher (http://localhost:3000/__cypress/runner/cypress_runner.js:11318:23)
      at Promise._settlePromiseFromHandler (http://localhost:3000/__cypress/runner/cypress_runner.js:9253:31)
      at Promise._settlePromise (http://localhost:3000/__cypress/runner/cypress_runner.js:9310:18)
      at Promise._settlePromise0 (http://localhost:3000/__cypress/runner/cypress_runner.js:9355:10)
      at Promise._settlePromises (http://localhost:3000/__cypress/runner/cypress_runner.js:9431:18)
      at _drainQueueStep (http://localhost:3000/__cypress/runner/cypress_runner.js:6025:12)
      at _drainQueue (http://localhost:3000/__cypress/runner/cypress_runner.js:6018:9)
      at ../../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:3000/__cypress/runner/cypress_runner.js:6034:5)
      at Async.drainQueues (http://localhost:3000/__cypress/runner/cypress_runner.js:5904:14)
  From Your Spec Code:
      at Context.eval (http://localhost:3000/__cypress/tests?p=tests/support/index.js:487:13)
  
  From Node.js Internals:
    TimeoutError: page.waitForNavigation: Timeout 30000ms exceeded.
    =========================== logs ===========================
    waiting for navigation until "load"
    ============================================================
    page.waitForNavigation: Timeout 30000ms exceeded.
    =========================== logs ===========================
    waiting for navigation until "load"
    ============================================================
        at Object.waitAndClick (/workspaces/arb-token-bridge/node_modules/@synthetixio/synpress/commands/playwright.js:162:16)
        at Object.acceptAccess (/workspaces/arb-token-bridge/node_modules/@synthetixio/synpress/commands/metamask.js:591:5)
        at acceptMetamaskAccess (/workspaces/arb-token-bridge/node_modules/@synthetixio/synpress/plugins/index.js:183:24)

[drptbl]

Hey @fionnachan, thanks for report!

I will investigate this today.

Thanks,
Jakub.

[roninjin10]

I'm having same issue runing cypress in xvfb on 2.3.2

[drptbl]

@roninjin10 It also works locally but fails on Codespaces? Could you please upload a video from CI? Also, a sample failing e2e test would be very welcome 🙏.

[roninjin10]

No not using code spaces. Just github actions with xvfb. Unfortunately we are still in process of open sourcing the optimism gateway. I might be able to ask to give you access though.

This is the github action I am running

The test file we are running is here https://gist.github.com/roninjin10/69ceae736d1f4122c37c8fe9199a6a0c

Version is synpress 2.3.2

E2E:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Setup
        uses: ./.github/actions/setup
      - name: Synpress E2E Tests
        uses: GabrielBB/xvfb-action@v1
        with:
          run: yarn e2e:ci
        env:
          SECRET_WORDS: ${{ secrets.E2E_KOVAN_SEED }}
          CYPRESS_CACHE_FOLDER: ${{ github.workspace }}/node_modules/.cache/Cypress
          CYPRESS_RESOURCES_WAIT: 1

      - name: Archive e2e artifacts
        uses: actions/upload-artifact@3cea5372237819ed00197afe530f5a7ea3e805c8 # pin@v2
        if: always()
        with:
          name: e2e-artifacts
          retention-days: 3
          path: |
            packages/frontend/tests/e2e/videos
            packages/frontend/tests/e2e/screenshots
        continue-on-error: true

[roninjin10]

This might be easy for me to see what is happening if you know of a quick easy way to also record the metamask window in my githubaction @drptbl

[drptbl]

@roninjin10 you can do it, but I'm not sure if that's as easy as it should be. You would have to do this through docker-compose.yml file.

--

Github Actions setup:

1. You would have to create Dockerfile for your project which will be a container prepared for running e2e tests inside:
   https://github.com/Synthetixio/synpress/blob/dev/Dockerfile
   It copies your project inside docker container and installs dependencies.
2. Create .dockerignore:
   https://github.com/Synthetixio/synpress/blob/dev/.dockerignore
3. Then you have to use this docker-compose.yml to run e2e tests:
   https://github.com/Synthetixio/synpress/blob/dev/docker-compose.yml
   Things to change:
   • environmental variables (if your tests require them): https://github.com/Synthetixio/synpress/blob/dev/docker-compose.yml#L7
   • path to your previously created Dockerfile (if it's not in the root dir of your project): https://github.com/Synthetixio/synpress/blob/dev/docker-compose.yml#L6
4. Run your tests with docker-compose on CI:
   https://github.com/Synthetixio/synpress/blob/dev/.github/workflows/audit_and_lint.yml#L92-L135

--

There is another option.. you can simply try to run everything inside docker container on your localhost, it will also record a video and maybe it will fail for you because it's similar environment to Github Actions.

You have to follow steps 1 and 2 from above, but instead of integrating all of this with CI (skip step 3), just run this command on your localhost to run your tests in docker container locally:
ENVVAR=something-if-you-need-it docker-compose up --build --exit-code-from synpress

Now, when your tests run, you can connect to http://localhost:8080/vnc.html?autoconnect=true to see what is going on with your e2e tests inside docker container in real time and interact with them through browser.

After tests are finished, video from the container with metamask will be saved on your local machine, in your root project folder, at docker/videos-ci path.

[drptbl]

I'm still working on this issue, so all informations are welcome 🙏. Thanks for helping me out!

[rori4]

@drptbl we are also facing this issue as well inside GH actions with @synthetixio/synpress "2.3.2"

I can tell you that the tests we did fail with

cy.task('acceptMetamaskAccess') failed with the following error:
waiting for function failed: timeout 30000ms exceeded

also there was a console error that bubbled in the test

"Request of type 'wallet_requestPermissions' already pending for origin http://localhost:3000. Please wait."

I did try to switch @synthetixio/synpress: "2.0.1", and it resolved the issue but then we had another one there with setupMetamask timeout-ing on consecutive tests

I hope that helps

[drptbl]

@rori4 is there any example repo or code where I can reproduce this issue? I'm still missing a full example of working code where I can reproduce this issue.

[rori4]

@drptbl I will try to re-create the issue in example repo when I have time but idk when I will have that. In the meantime check telegram

[drptbl]

@roninjin10 I have just seen your CI setup from above:
#498 (comment)

I think this is the issue. Display is not properly emulated (or attached) and synpress is having troubles. Could you please try to change your CI setup to an official, recommended way using docker-compose.yml and see if that helps you?

I have described everything here:
#498 (comment)
https://github.com/Synthetixio/synpress#-using-with-docker

Also, have you seen "CI tips & tricks"?
https://github.com/Synthetixio/synpress/#ci-tips--tricks
It mentions few things.. for example that resolution should be quite high (I'm sure it's not in your setup and this could be an issue) and docker-e2e is recommended container to run tests because it contains everything that is needed. If you want to use your own container, you should make sure that it contains all dependencies required.

cc @rori4 @fionnachan
I would love to get a feedback to see if this solution helps for anyone else.

^ if this is the issue, I will try few things:

1. Find a way to run synpress on lower resolutions
2. Make it easier to run synpress without official docker-compose.yml stack
3. Improve error logging (and add some checks) for this specific case

Just need a confirmation that this is it. If you want, I can assist you in the setup, just link me (or add me) to your repository with failing e2e tests.

Tyvm for your help on debugging this issue 🙏.

[roninjin10]

Got it. Based on your advice, I'm going to first really quickly try passing in --server-args="-screen 0, 1920x1080x24" into xvfb and see if the larger screen fixes the issue (the default is 1024x768x24). If that works I'll report back. Otherwise I will try using docker later this week.

I have a feeling we will want to use docker even it it works because recording the metamask window is a killer upgrade to our current setup.

[roninjin10]

Giving xvfb a larger screen did not fix the issue. My next step was to see if running xvfb locally would produce same result but it turns out it's nearly impossible to run xvfb on a mac and doesn't seem worth it to spin up a linux box just for this test. Trying to set up docker-compose build next.

[drptbl]

@roninjin10 @rori4 @fionnachan

Hey,
just wanted to give you an update on this issue.

First of all, @rori4 has reported that my previous recommendation of using official docker-compose.yml has fixed issues for him. I have described all steps here:
#498 (comment)
My guess is that the issue was related to too low resolution (which synpress didn't support before) or emulated display was not being properly attached to synpress (which is mandatory because extensions work only in non-headless mode; and this is properly done inside official docker-compose.yml).

Other than that, I have just released @synthetixio/synpress@2.4.0 which adds support for lower resolutions. I encourage you to try it as it may fix some of your issues.

Now, if you're still experiencing issues - it will be mostly related to emulated display being not properly configured/attached in your setup. Please use docker-compose.yml as a reference.

If you're still experiencing any issues related to this thread, please re-open it and let me know. I'm also looking forward to your feedback which is super-helpful in development of synpress.

Big thanks 🙏,
Jakub.