Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: upgrade tests #344

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

dkaras-splunk
Copy link

TA-> TA Upgrade implementation ADDON-73792

This pull request introduces a new feature to our CI/CD pipeline, focusing on automating upgrade testing for Technology Add-ons (TA). The goal is to streamline the release process by incorporating scalable and user-friendly solutions capable of executing upgrade scenarios

Key Features:
Automated Upgrade Testing: Introduces jobs for upgrade testing within the GitHub CI/CD pipeline.

Flexible Version Testing: Allows users to provide multiple TA versions on which upgrade tests should be executed. In this repo, ta_upgrade_version parameter is responsible for passing the info about versions being tested. Also, both GitHub and Splunkbase releases are supported, depending on the format of the version string provided:

vX.X.X - for GitHub releases
X.X.X - for Splunkbase releases
additionally, latest can be passed to use the latest version from GitHub
example run - https://github.com/splunk/splunk-add-on-for-amazon-web-services/actions/runs/12045945177

Part of
https://github.com/splunk/ta-automation-app-of-apps/pull/27
https://github.com/splunk/ta-automation-k8s-manifests/pull/102
splunk/wfe-test-runner-action#35

@awownysz-splunk
Copy link
Contributor

@awownysz-splunk awownysz-splunk force-pushed the feat/ADDON-73868-implement-upgrade-tests branch from 355238a to ea4a921 Compare December 9, 2024 10:36
Co-authored-by: Adam Wownysz <awownysz@splunk.com>
@awownysz-splunk awownysz-splunk force-pushed the feat/ADDON-73868-implement-upgrade-tests branch from ea4a921 to 771d953 Compare December 9, 2024 11:57
@awownysz-splunk
Copy link
Contributor

awownysz-splunk commented Dec 9, 2024

Additional details about changes in this repo:

  • upgrade-tests-ta-versions - new input to the reusable workflow that needs to be provided in each TA that aims to run upgrade tests. This is a list of TA versions for which the upgrade tests will be executed. This list, together with supported Splunk versions (already embedded in the workflow) defines the matrix of upgrade test jobs. A default value is latest, which means using latest release of the TA on GitHub.
  • run-upgrade-tests - new job in the workflow which defines the execution of upgrade tests in GitHub Actions. As of now, upgrade tests are executed only when assigning execute_upgrade label to the PR (also for PRs to main). Similarly to all other tests (run-ui-tests, run-modinput-tests, etc.), it triggers the execution of Argo workflow for each job within the generated matrix ('TA version - Splunk version' in this case).

pawel-dabro
pawel-dabro previously approved these changes Dec 13, 2024
@kdoroszko-splunk
Copy link
Contributor

Sometime ago we added test summary jobs for each test type. Would you consider adding that as well?
Here are two PRs that could be helpful: feat PR, fix PR

@awownysz-splunk
Copy link
Contributor

Sometime ago we added test summary jobs for each test type. Would you consider adding that as well? Here are two PRs that could be helpful: feat PR, fix PR

upgrade-tests-report added now

@@ -675,7 +698,13 @@ jobs:
name: package-deployment
path: build/package/deployment**
if: ${{ !cancelled() }}

- name: app-id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is not necessary after switching logic of accessing packages from splunkbase to aws s3.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

vendor-version: ${{ matrix.vendor-version.image }}
sc4s-version: "No"
k8s-manifests-branch: ${{ needs.setup.outputs.k8s-manifests-branch }}
app-id: ${{ needs.build.outputs.app-id }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

app-id is no longer necessary

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

@awownysz-splunk awownysz-splunk force-pushed the feat/ADDON-73868-implement-upgrade-tests branch from a4d72db to 44dbcef Compare December 17, 2024 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants