Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(smoke-test): Arm "test installers" GHA workflow #6718

Merged
merged 7 commits into from
Feb 14, 2025

Conversation

kraenhansen
Copy link
Contributor

@kraenhansen kraenhansen commented Feb 12, 2025

Here's a link for a successful run of the workflow: https://github.com/mongodb-js/compass/actions/runs/13292993666

Description

Merging this PR will:

Add the e2e-tests as a dependency of smoke-tests

72b40f2 adds the e2e-tests as a dependency of the smoke-tests package

"compass-e2e-tests": "^1.29.0",

I'm doing this to leverage Lerna to run "bootstrap" only for the packages that we actually need to run the test:

- name: Install dependencies and build packages
run: npm ci
- name: Bootstrap packages
run: npx lerna run bootstrap --scope @mongodb-js/compass-smoke-tests --include-dependencies

Arm the "test installers" workflow

Running the smoke tests (time-to-first-query, auto-update-from and auto-update-to) with various package formats, platforms (MacOS x86 + arm64, Ubuntu Linux x86 and Windows x86) and variants / distributions (regular, read-only and isolated).

For now, I'm disabling a few of these combinations to get a successful run that I can extend to trigger from Evergreen (or some other event) in a future PR:

exclude:
# TODO: See https://github.com/mongodb-js/compass/actions/runs/13281152689/job/37079620322
- package: osx_dmg
test: auto-update-from
# TODO: See https://github.com/mongodb-js/compass/actions/runs/13281152689/job/37079621700
- package: windows_setup
test: auto-update-from
# Skip auto-update tests for isolated and readonly distributions for now
# This is not supported by the update server we're using in the test harness
- hadron-distribution: compass-isolated
test: auto-update-from
- hadron-distribution: compass-isolated
test: auto-update-to
- hadron-distribution: compass-readonly
test: auto-update-from
- hadron-distribution: compass-readonly
test: auto-update-to
# Temporary skip failing auto-update-to tests
- test: auto-update-to
package: osx_dmg
- test: auto-update-to
package: windows_zip
- test: auto-update-to
package: windows_msi
- test: auto-update-to
package: windows_setup
- test: auto-update-to
package: linux_deb
# Skip time-to-first-query tests for readonly because it doesn't have editable documents
# See https://github.com/mongodb-js/compass/actions/runs/13286945911/job/37097791601
- hadron-distribution: compass-readonly
test: time-to-first-query

Refactor the auto-update-to tests

The is introducerd in refactors the auto-update-to tests. The main issue was that the installer for the "subject" of the test was being used to install the existing "latest" package. This was an issue for example on MacOS where we use a "latest" .dmg to test the "subject" .zip release. I didn't notice this before adding assertions on the kind passed to the installer.

Running cleanup on exit, warning on failures

If for some reason some process is keeping files alive in a sandbox directory, Windows was failing the test job even when the actual test succeeded.

5d1188a introduce a warning instead of throwing, as we consider the disks on CI ephemeral anyway. It also registers a listener on the process exiting or getting interrupted to run the clean-up, to make it more gentle on outstanding async work and interruptions while running.

Waiting for auto-update server to close

I had a small suspicion that there that not awaiting the shutdown of the auto-updater server was causing a race condition, but I don't think that's the case. I left in the code to await it closing anyway, in 42d8eb8

Checklist

Motivation and Context

  • Bugfix
  • New feature
  • Dependency update
  • Misc

Open Questions

Dependents

Types of changes

  • Backport Needed
  • Patch (non-breaking change which fixes an issue)
  • Minor (non-breaking change which adds functionality)
  • Major (fix or feature that would cause existing functionality to change)

@kraenhansen kraenhansen self-assigned this Feb 12, 2025
@kraenhansen kraenhansen requested a review from lerouxb February 12, 2025 19:52
@kraenhansen kraenhansen added no release notes Fix or feature not for release notes no-title-validation Skips validation of PR titles (conventional commit adherence + JIRA ticket inclusion) labels Feb 13, 2025
@kraenhansen kraenhansen force-pushed the kh/test-installers-on-gha-armed branch from 42d8eb8 to eb0bf05 Compare February 13, 2025 13:19
@kraenhansen kraenhansen force-pushed the kh/test-installers-on-gha-armed branch from eb0bf05 to 9d6e021 Compare February 13, 2025 18:47
@kraenhansen kraenhansen merged commit 7682e06 into main Feb 14, 2025
35 checks passed
@kraenhansen kraenhansen deleted the kh/test-installers-on-gha-armed branch February 14, 2025 07:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no release notes Fix or feature not for release notes no-title-validation Skips validation of PR titles (conventional commit adherence + JIRA ticket inclusion)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants