Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unskip tests for chrome, fix tags #158405

Merged
merged 2 commits into from
May 26, 2023
Merged

unskip tests for chrome, fix tags #158405

merged 2 commits into from
May 26, 2023

Conversation

dmlemeshko
Copy link
Member

@dmlemeshko dmlemeshko commented May 24, 2023 •
edited by kibanamachine
Loading

Summary

With Firefox update to v113 on our CI workers, we noticed that tests are run slower and more often fail with timeouts.
Unfortunately our auto-skip functionality skip the test suite completely, meaning it won't run on Chrome as well.

This PR unskips the firefox failed test to run on Chrome, I also fix the labels for some suites to run only sub set of tests for now.

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
test/functional/apps/visualize/group5/_tsvb_time_series.ts
Comment on lines +54 to 55
describe('basics', function () {
this.tags('includeFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not obvious, but with arrow function tags are not applied.

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
test/functional/apps/visualize/group5/_tsvb_time_series.ts
@@ -594,7 +595,8 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
after(async () => await visualBuilder.toggleNewChartsLibraryWithDebug(false));
});

describe('index pattern selection mode', () => {
describe('index pattern selection mode', function () {
this.tags('skipFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think originally intention was to run only basics suite on FF. Now we are limited with slowness, I suggest we stick to the minimum until we find a solution.

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
x-pack/test/functional/apps/infra/home_page.ts Outdated
this.tags('includeFirefox');
describe('Home page', function () {
// Failing: See https://github.com/elastic/kibana/issues/157713
// this.tags('includeFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unskipping for Chorme

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
x-pack/test/functional/apps/infra/metrics_explorer.ts
Comment on lines +103 to +105
describe('Saved Views', function () {
// FLAKY: https://github.com/elastic/kibana/issues/157738
this.tags('skipFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unskipping for Chrome

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
x-pack/test/functional/apps/security/security.ts
Comment on lines +19 to +21
describe('Security', function () {
// FLAKY: https://github.com/elastic/kibana/issues/157722
// this.tags('includeFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unskipping for Chrome

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
x-pack/test/functional/apps/spaces/spaces_selection.ts
@@ -127,7 +127,8 @@ export default function spaceSelectorFunctionalTests({
});
});

describe('Spaces Data', () => {
describe('Spaces Data', function () {
this.tags('skipFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are limited with FF slowness right now, I suggest we stick to the minimum tests on FF until we find a solution.

dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
x-pack/test/functional/apps/watcher/watcher_test.js
Comment on lines +23 to +25
describe('watcher_test', function () {
// FLAKY: https://github.com/elastic/kibana/issues/157723
// this.tags('includeFirefox');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unskipping for Chrome

@dmlemeshko dmlemeshko requested a review from a team May 24, 2023 17:45
@dmlemeshko dmlemeshko force-pushed the review-ftr-firefox-tests branch from 69068df to 3023a93 Compare May 24, 2023 18:45
dmlemeshko
dmlemeshko commented May 24, 2023
View reviewed changes
x-pack/test/functional/apps/infra/home_page.ts
@@ -22,6 +22,7 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
const testSubjects = getService('testSubjects');

// Failing: See https://github.com/elastic/kibana/issues/157713
// Fails on both Chrome and Firefox
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to unskip for Chrome, but got failure 2 times in a row

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
securitySolution 398 402 +4
total +6

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
securitySolution 478 482 +4
total +6

History

  • 💔 Build #129991 failed ee92ebc819c46a3db7b7485ff2acc8a9aebf9e39

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@dmlemeshko dmlemeshko added release_note:skip Skip the PR/issue when compiling release notes v8.9.0 v8.8.1 labels May 24, 2023
@dmlemeshko dmlemeshko marked this pull request as ready for review May 24, 2023 21:38
@dmlemeshko dmlemeshko requested review from a team as code owners May 24, 2023 21:38
stratoula
stratoula approved these changes May 25, 2023
View reviewed changes
Copy link
Contributor

@stratoula stratoula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Visualizations team changes LGTM, code review only

thomheymann
thomheymann approved these changes May 25, 2023
View reviewed changes
Copy link
Contributor

@thomheymann thomheymann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR - LGTM!

@dmlemeshko dmlemeshko enabled auto-merge (squash) May 25, 2023 12:17
@jeramysoucy jeramysoucy mentioned this pull request May 25, 2023
Closed
pheyos
pheyos approved these changes May 26, 2023
View reviewed changes
Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

mohamedhamed-ahmed
mohamedhamed-ahmed approved these changes May 26, 2023
View reviewed changes
Copy link
Contributor

@mohamedhamed-ahmed mohamedhamed-ahmed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Infra changes LGTM!

@dmlemeshko dmlemeshko merged commit 5d4eec5 into elastic:main May 26, 2023
@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
❌ 8.8 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 158405

Questions ?

Please refer to the Backport tool documentation

@dmlemeshko dmlemeshko mentioned this pull request May 26, 2023
Merged
@dmlemeshko
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
✅ 8.8

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

dmlemeshko added a commit to dmlemeshko/kibana that referenced this pull request May 26, 2023
@dmlemeshko
unskip tests for chrome, fix tags (elastic#158405)
b8ae4f1 
## Summary

With Firefox update to v113 on our CI workers, we noticed that tests are
run slower and more often fail with timeouts.
Unfortunately our auto-skip functionality skip the test suite
completely, meaning it won't run on Chrome as well.

This PR unskips the firefox failed test to run on Chrome, I also fix the
labels for some suites to run only sub set of tests for now.

(cherry picked from commit 5d4eec5)

# Conflicts:
#	x-pack/test/functional/apps/infra/home_page.ts
#	x-pack/test/functional/apps/infra/metrics_explorer.ts
dmlemeshko added a commit that referenced this pull request May 26, 2023
@dmlemeshko
[8.8] unskip tests for chrome, fix tags (#158405) (#158545)
dc74b99 
# Backport

This will backport the following commits from `main` to `8.8`:
- [unskip tests for chrome, fix tags
(#158405)](#158405)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2023-05-26T08:19:53Z","message":"unskip
tests for chrome, fix tags (#158405)\n\n## Summary\r\n\r\nWith Firefox
update to v113 on our CI workers, we noticed that tests are\r\nrun
slower and more often fail with timeouts.\r\nUnfortunately our auto-skip
functionality skip the test suite\r\ncompletely, meaning it won't run on
Chrome as well.\r\n\r\nThis PR unskips the firefox failed test to run on
Chrome, I also fix the\r\nlabels for some suites to run only sub set of
tests for
now.","sha":"5d4eec5131a797708bd2b8fae5a4706ecc3e7679","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.9.0","v8.8.1"],"number":158405,"url":"https://github.com/elastic/kibana/pull/158405","mergeCommit":{"message":"unskip
tests for chrome, fix tags (#158405)\n\n## Summary\r\n\r\nWith Firefox
update to v113 on our CI workers, we noticed that tests are\r\nrun
slower and more often fail with timeouts.\r\nUnfortunately our auto-skip
functionality skip the test suite\r\ncompletely, meaning it won't run on
Chrome as well.\r\n\r\nThis PR unskips the firefox failed test to run on
Chrome, I also fix the\r\nlabels for some suites to run only sub set of
tests for
now.","sha":"5d4eec5131a797708bd2b8fae5a4706ecc3e7679"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/158405","number":158405,"mergeCommit":{"message":"unskip
tests for chrome, fix tags (#158405)\n\n## Summary\r\n\r\nWith Firefox
update to v113 on our CI workers, we noticed that tests are\r\nrun
slower and more often fail with timeouts.\r\nUnfortunately our auto-skip
functionality skip the test suite\r\ncompletely, meaning it won't run on
Chrome as well.\r\n\r\nThis PR unskips the firefox failed test to run on
Chrome, I also fix the\r\nlabels for some suites to run only sub set of
tests for
now.","sha":"5d4eec5131a797708bd2b8fae5a4706ecc3e7679"}},{"branch":"8.8","label":"v8.8.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
This was referenced Jun 23, 2023
Closed
Merged
@alisonelizabeth alisonelizabeth mentioned this pull request Aug 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stratoula stratoula stratoula approved these changes

@mohamedhamed-ahmed mohamedhamed-ahmed mohamedhamed-ahmed approved these changes

@pheyos pheyos pheyos approved these changes

@thomheymann thomheymann thomheymann approved these changes

Assignees
No one assigned
Labels
release_note:skip Skip the PR/issue when compiling release notes v8.8.1 v8.9.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@dmlemeshko @kibana-ci @kibanamachine @thomheymann @pheyos @mohamedhamed-ahmed @stratoula