Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Add tests to github workflow #980

Closed
wants to merge 1 commit into from

Conversation

kavilla
Copy link
Member

@kavilla kavilla commented Nov 27, 2021

Description

Add unit tests to github workflow and also creating a "bad apples"
environment variable. Some unit tests just fail on the CI for
hardware issues. They should be improved but step one will be
calling out the bad apples.

Also due to the flakiness we can cache the previous run results
and only run the tests that failed. It's too random to catch
with the bad apples mechanism. But still added the continue on
error for unit tests because it takes so long to re-run on the
CI. So instead if it does fail we automatically comment there
was a failure and ask them to re-run.

Next step will be improving.

Also needed to limit the amount of workers because otherwise the
hardware can't handle well so then it will accidentally create conflicts.
This means we get an accurate test run but it is slower on the CI.

Included integration tests which worked out of the box.

Included e2e tests as well but it the chrome driver for the application
was different from github's chrome so to run it I just upgraded it for
the test run. Not ideal, ideally we should probably set up a
docker env and install the specific versions since we are now
depending on github's virtual env and the dependencies they installed
there. But at least this is a first pace.

Signed-off-by: Kawika Avilla kavilla414@gmail.com

Issues Resolved

n/a

Check List

  • New functionality includes testing.
    • All tests pass
      • yarn test:jest
      • yarn test:jest_integration
      • yarn test:ftr
  • New functionality has been documented.
  • Commits are signed per the DCO using --signoff

Add unit tests to github workflow and also creating a "bad apples"
environment variable. Some unit tests just fail on the CI for
hardware issues. They should be improved but step one will be
calling out the bad apples.

Also due to the flakiness we can cache the previous run results
and only run the tests that failed. It's too random to catch
with the bad apples mechanism. But still added the continue on
error for unit tests because it takes so long to re-run on the
CI. So instead if it does fail we automatically comment there
was a failure and ask them to re-run.

Next step will be improving.

Also needed to limit the amount of workers because otherwise the
hardware can't handle well so then it will accidentally create conflicts.
This means we get an accurate test run but it is slower on the CI.

Included integration tests which worked out of the box.

Included e2e tests as well but it the chrome driver for the application
was different from github's chrome so to run it I just upgraded it for
the test run. Not ideal, ideally we should probably set up a
docker env and install the specific versions since we are now
depending on github's virtual env and the dependencies they installed
there. But at least this is a first pace.

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
@kavilla kavilla requested a review from a team as a code owner November 27, 2021 08:04
@kavilla
Copy link
Member Author

kavilla commented Nov 27, 2021

Variant of this PR: #906

Except this one differs by allowing re-runs if failures and will only run the specific steps that failed. Please note, if someone pushing again it will create a new job and re-run all the steps. The biggest con about this is that it will basically not logged the steps that were successful in previous runs.

If you check out this run: https://github.com/kavilla/OpenSearch-Dashboards-1/actions/runs/1509682521

you can see the first run the unit tests were flaky, then the second run was a re-run and it didn't re-run the previous successful steps just the build and unit tests.

@kavilla kavilla closed this Nov 27, 2021
AMoo-Miki pushed a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Feb 10, 2022
AMoo-Miki pushed a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Feb 10, 2022
# [24.5.0](elastic/elastic-charts@v24.4.0...v24.5.0) (2021-01-30)

### Bug Fixes

* add theme min radius to point shape ([opensearch-project#996](elastic/elastic-charts#996)) ([98089a9](elastic/elastic-charts@98089a9))
* align tooltip z-index to EUI tooltip z-index ([opensearch-project#931](elastic/elastic-charts#931)) ([f7f1f6f](elastic/elastic-charts@f7f1f6f))
* chart state and series functions cleanup ([opensearch-project#989](elastic/elastic-charts#989)) ([42a7af0](elastic/elastic-charts@42a7af0))
* create unique ids for dot icons ([opensearch-project#971](elastic/elastic-charts#971)) ([0b3e00f](elastic/elastic-charts@0b3e00f))
* external tooltip legend extra value sync ([opensearch-project#993](elastic/elastic-charts#993)) ([7e1096e](elastic/elastic-charts@7e1096e))
* **legend:** disable focus and keyboard navigation for legend in partition ch… ([opensearch-project#952](elastic/elastic-charts#952)) ([dfff3e2](elastic/elastic-charts@dfff3e2))
* **legend:** hierarchical legend order should follow the tree paths ([opensearch-project#947](elastic/elastic-charts#947)) ([7b70186](elastic/elastic-charts@7b70186)), closes [opensearch-project#944](elastic/elastic-charts#944)
* **legend:** remove ids for circles ([opensearch-project#973](elastic/elastic-charts#973)) ([ed98481](elastic/elastic-charts@ed98481))

### Features

* **cursor:** improve theme styling for crosshair ([opensearch-project#980](elastic/elastic-charts#980)) ([0248ad6](elastic/elastic-charts@0248ad6))
* **legend:**  display pie chart legend extra ([opensearch-project#939](elastic/elastic-charts#939)) ([672a4df](elastic/elastic-charts@672a4df))
* **legend:** add keyboard navigation ([opensearch-project#880](elastic/elastic-charts#880)) ([b471a94](elastic/elastic-charts@b471a94))
* **partition:** Flame and icicle chart ([opensearch-project#965](elastic/elastic-charts#965)) ([9e8b1f7](elastic/elastic-charts@9e8b1f7))
* **partition:** legend hover options ([opensearch-project#978](elastic/elastic-charts#978)) ([acd1339](elastic/elastic-charts@acd1339))
* **xy:** support multiple point shapes on line, area and bubble charts ([opensearch-project#988](elastic/elastic-charts#988)) ([4f23b4f](elastic/elastic-charts@4f23b4f))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant