[Bug]: Cypress unexpectedly exited when running OSD integ test

[tianleh]

Describe the bug

When running OSD integ test, either from cli ./test.sh integ-test manifests/1.3.0/opensearch-dashboards-1.3.0-test.yml or Jenkins. We can see that the test suite failed because of the following error.

The Test Runner unexpectedly exited via a exit event with signal SIGABRT

Please search Cypress documentation for possible solutions:

https://on.cypress.io

Check if there is a GitHub issue describing this crash:

https://github.com/cypress-io/cypress/issues

Consider opening a new issue.

----------

Platform: linux (CentOS - 7.9.2009)
Cypress Version: 5.6.0

To reproduce

locally run

./test.sh integ-test manifests/1.3.0/opensearch-dashboards-1.3.0-test.yml

or rerun Jenkins job
http://jenki-jenki-fpgmrv2ryxko-1366042710.us-east-1.elb.amazonaws.com/job/integ-test-opensearch-dashboards/
See the following sample output. Scroll all the way to the bottom.

https://ci.opensearch.org/ci/dbc/integ-test-opensearch-dashboards/1.3.0/1484/linux/x64/test-results/b94f426463b24046a2ed33c5be290508/integ-test/functionalTestDashboards/without-security/test-results/stdout.txt

Expected behavior

No response

Screenshots

If applicable, add screenshots to help explain your problem.

Host / Environment

No response

Additional context

No response

Relevant log output

No response

[tianleh]

Looks like it is because of the default browser Electron cypress-io/cypress#5965

In FTRpo's CI, the same test cases are passing with browser Chrome. https://github.com/opensearch-project/opensearch-dashboards-functional-test/actions/runs/1936788918

[tianleh]

Possible directions:

1. install Chrome in infra's Docker image https://hub.docker.com/layers/opensearchstaging/ci-runner/centos7-x64-arm64-jdkmulti-node10.24.1-cypress6.9.1-20211130/images/sha256-d5a892e7d6b824e2b33ab4e28b88c7baa2d2573095163d3da92e9ab074993be9?context=explore

2. check more configuration parameters of Cypress to tune the situation

[tianleh]

After the FTRepo changed Github action to use default browser, it has the same error. https://github.com/opensearch-project/opensearch-dashboards-functional-test/runs/5430672181?check_suite_focus=true

/home/runner/.cache/Cypress/5.6.0/Cypress/Cypress[2277]: ../../third_party/electron_node/src/api/callback.cc:226:MaybeLocal<v8::Value> node::MakeCallback(v8::Isolate *, Local<v8::Object>, Local<v8::Function>, int, Local<v8::Value> *, node::async_context): Assertion `(env) != nullptr' failed.
The Test Runner unexpectedly exited via a exit event with signal SIGABRT

Please search Cypress documentation for possible solutions:

https://on.cypress.io/

Check if there is a GitHub issue describing this crash:

https://github.com/cypress-io/cypress/issues

Consider opening a new issue.

[tianleh]

Possible directions:

1. install Chrome in infra's Docker image https://hub.docker.com/layers/opensearchstaging/ci-runner/centos7-x64-arm64-jdkmulti-node10.24.1-cypress6.9.1-20211130/images/sha256-d5a892e7d6b824e2b33ab4e28b88c7baa2d2573095163d3da92e9ab074993be9?context=explore

2. check more configuration parameters of Cypress to tune the situation

@peterzhuamazon If having Chrome installed in the Docker image is easy to do, I would prefer taking this approach.

[tianleh]

cc @seraphjiang

[tianleh]

The download feature may be the reason why Electron failed.

[tianleh]

In FTRepo, the tests are successfully by running against Chrome https://github.com/opensearch-project/opensearch-dashboards-functional-test/runs/5437424313?check_suite_focus=true

[tianleh]

Research a bit. Find that it is feasible to install Chrome for x86 64
curl https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm

However, I could not find such for arm 64. Looks like the report plugin has handled such https://github.com/opensearch-project/dashboards-reports/blob/main/.github/workflows/dashboards-reports-test-and-build-workflow.yml

@joshuali925 could you please share some experiences here?

cc @seraphjiang @peterzhuamazon

[tianleh]

In FTRepo's Github action workflow, the tests are stably passing https://github.com/opensearch-project/opensearch-dashboards-functional-test/runs/5476767197?check_suite_focus=true

[joshuali925]

chrome doesn't have arm64 build, need to use chromium. The reporting headless chromium is built from source and has arm64 version so you can try it. details see https://github.com/opensearch-project/dashboards-reports/tree/main/dashboards-reports/rendering-engine/headless-chrome

[tianleh]

Some existing error in Electron

npm WARN cypress-multi-reporters@1.5.0 requires a peer of mocha@>=3.1.2 but none is installed. You must install peer dependencies yourself.
npm WARN mocha-junit-reporter@2.0.0 requires a peer of mocha@>=2.2.5 but none is installed. You must install peer dependencies yourself.

Couldn't determine Mocha version
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db

Why you should do it regularly:
https://github.com/browserslist/browserslist#browsers-data-updating
Couldn't determine Mocha version
(node:825804) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 preprocessor:close listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
error Command failed with exit code 6.

[joshuali925]

Adding instructions to run against reporting chromium just for reference:

1. unzip reporting chromium
   • amd64: https://github.com/opensearch-project/dashboards-reports/releases/download/chromium-1.12.0.0/chromium-linux-x64.zip
   • arm64: https://github.com/opensearch-project/dashboards-reports/releases/download/chromium-1.12.0.0/chromium-linux-arm64.zip
2. Add to cypress/plugins/index.js

module.exports = (on, config) => {
  return {
    browsers: config.browsers.concat({
      name: 'headless_shell',
      channel: 'stable',
      family: 'chromium',
      displayName: 'headless_shell',
      version: '77.0.3844.0',
      path: '.chromium/headless_shell',
      majorVersion: '77'
    }),
  }
}

3. Run cypress with --browser headless_shell