Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E non-knownflake fail should print prior error #1298

Open
kaovilai opened this issue Jan 31, 2024 · 4 comments
Open

E2E non-knownflake fail should print prior error #1298

kaovilai opened this issue Jan 31, 2024 · 4 comments
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. testing

Comments

@kaovilai
Copy link
Member

Flake fail should print why tests fail and why it's being retried from prior run instead of "No known FLAKE found in a previous run, marking test as failed."

It should save a prior error and print out during Fail() call

	DescribeTable("Backup and restore applications",
		func(brCase BackupRestoreCase, expectedErr error) {
			if CurrentSpecReport().NumAttempts > 1 && !knownFlake {
				Fail("No known FLAKE found in a previous run, marking test as failed.") // <--
			}
@shubham-pampattiwar
Copy link
Member

++

@mateusoliveira43
Copy link
Contributor

The request is partially wrong. When code enters if, it fails immediately, which is the case when knownFlake is false, which means none of these https://github.com/openshift/oadp-operator/blob/master/tests/e2e/lib/flakes.go#L38-L49 occurred. Example of flake happening and being retryed https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_oadp-operator/1294/pull-ci-openshift-oadp-operator-master-4.12-e2e-test-aws/1752417708428759040/artifacts/e2e-test-aws/e2e/build-log.txt

Message "No known FLAKE found in a previous run, marking test as failed." only appears when none of the items in the list happened, and test fails immediately. Example of this behavior https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_oadp-operator/1287/pull-ci-openshift-oadp-operator-master-4.13-e2e-test-aws/1751242762205794304/artifacts/e2e-test-aws/e2e/build-log.txt

@mpryc for your information

@kaovilai kaovilai changed the title E2E Unknown flake fail should print prior error E2E non-flake fail should print prior error Jan 31, 2024
@kaovilai kaovilai changed the title E2E non-flake fail should print prior error E2E non-knownflake fail should print prior error Jan 31, 2024
@openshift-bot
Copy link

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 1, 2024
@kaovilai
Copy link
Member Author

kaovilai commented May 1, 2024

/lifecycle frozen

@openshift-ci openshift-ci bot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. testing
Projects
None yet
Development

No branches or pull requests

4 participants