[Security Solution] Some Prebuilt Rules Cypress tests don't run in CI #192256
Assignees
Labels
8.16 candidate
Team:Detection Rule Management
Security Detection Rule Management Team
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
test-coverage
issues & PRs for improving code test coverage
Comments
nikitaindik
added
8.16 candidate
Team: SecuritySolution
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Sep 13, 2024
… in CI (elastic#191978) **Resolves: elastic#192256 ## Summary This PR re-enables two Cypress test files that didn't run on CI: `update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also fixes failing tests in `prebuilt_rules_preview.cy.ts`. ### Changes - Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run on CI because it wasn't picked up by a glob [here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14). - `prebuilt_rules_preview.cy.ts`: - Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe` block instead of having it in a variable that is used in every `describe`. Apparently the tool we use to parse tags doesn't recognize tags in variables anymore, so this test didn't run in either ESS or Serverless pipelines. - Removed `describe('All environments' ... ` wrappers since they don't add any value anymore. Didn't remove any actual tests. - Reverted a change from this [PR](elastic#181427) that added a backdrop to the modal which doesn't allow user to switch rules without closing the modal. We have a [test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182) that checks that such switching is possible and this test started to fail once I reactivated the test file. - Fixed selectors that grab filters in the Overview tab. The old ones stopped working. Probably because of a change to the filters component that is built by another team. #### Correct behaviour: Switching between rules with flyout open https://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798 (cherry picked from commit c65c2ae)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Sep 13, 2024
… in CI (elastic#191978) **Resolves: elastic#192256 ## Summary This PR re-enables two Cypress test files that didn't run on CI: `update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also fixes failing tests in `prebuilt_rules_preview.cy.ts`. ### Changes - Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run on CI because it wasn't picked up by a glob [here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14). - `prebuilt_rules_preview.cy.ts`: - Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe` block instead of having it in a variable that is used in every `describe`. Apparently the tool we use to parse tags doesn't recognize tags in variables anymore, so this test didn't run in either ESS or Serverless pipelines. - Removed `describe('All environments' ... ` wrappers since they don't add any value anymore. Didn't remove any actual tests. - Reverted a change from this [PR](elastic#181427) that added a backdrop to the modal which doesn't allow user to switch rules without closing the modal. We have a [test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182) that checks that such switching is possible and this test started to fail once I reactivated the test file. - Fixed selectors that grab filters in the Overview tab. The old ones stopped working. Probably because of a change to the filters component that is built by another team. #### Correct behaviour: Switching between rules with flyout open https://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798 (cherry picked from commit c65c2ae)
nikitaindik
added a commit
to nikitaindik/kibana
that referenced
this issue
Sep 13, 2024
… in CI (elastic#191978) **Resolves: elastic#192256 ## Summary This PR re-enables two Cypress test files that didn't run on CI: `update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also fixes failing tests in `prebuilt_rules_preview.cy.ts`. ### Changes - Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run on CI because it wasn't picked up by a glob [here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14). - `prebuilt_rules_preview.cy.ts`: - Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe` block instead of having it in a variable that is used in every `describe`. Apparently the tool we use to parse tags doesn't recognize tags in variables anymore, so this test didn't run in either ESS or Serverless pipelines. - Removed `describe('All environments' ... ` wrappers since they don't add any value anymore. Didn't remove any actual tests. - Reverted a change from this [PR](elastic#181427) that added a backdrop to the modal which doesn't allow user to switch rules without closing the modal. We have a [test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182) that checks that such switching is possible and this test started to fail once I reactivated the test file. - Fixed selectors that grab filters in the Overview tab. The old ones stopped working. Probably because of a change to the filters component that is built by another team. #### Correct behaviour: Switching between rules with flyout open https://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798 (cherry picked from commit c65c2ae)
kibanamachine
referenced
this issue
Sep 13, 2024
…running in CI (#191978) (#192852) # Backport This will backport the following commits from `main` to `8.15`: - [[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)](#191978) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Nikita Indik","email":"nikita.indik@elastic.co"},"sourceCommit":{"committedDate":"2024-09-13T12:00:57Z","message":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)\n\n**Resolves: https://github.com/elastic/kibana/issues/192256**\r\n\r\n## Summary\r\n\r\nThis PR re-enables two Cypress test files that didn't run on CI:\r\n`update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also\r\nfixes failing tests in `prebuilt_rules_preview.cy.ts`.\r\n\r\n### Changes\r\n- Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run\r\non CI because it wasn't picked up by a glob\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14).\r\n - `prebuilt_rules_preview.cy.ts`:\r\n- Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe`\r\nblock instead of having it in a variable that is used in every\r\n`describe`. Apparently the tool we use to parse tags doesn't recognize\r\ntags in variables anymore, so this test didn't run in either ESS or\r\nServerless pipelines.\r\n- Removed `describe('All environments' ... ` wrappers since they don't\r\nadd any value anymore. Didn't remove any actual tests.\r\n- Reverted a change from this\r\n[PR](#181427) that added a\r\nbackdrop to the modal which doesn't allow user to switch rules without\r\nclosing the modal. We have a\r\n[test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182)\r\nthat checks that such switching is possible and this test started to\r\nfail once I reactivated the test file.\r\n- Fixed selectors that grab filters in the Overview tab. The old ones\r\nstopped working. Probably because of a change to the filters component\r\nthat is built by another team.\r\n\r\n\r\n#### Correct behaviour: Switching between rules with flyout open\r\n\r\nhttps://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798","sha":"c65c2ae4900a9f75db872cee056af35ff5f79cdc","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","backport:prev-major","v8.16.0"],"title":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI","number":191978,"url":"https://github.com/elastic/kibana/pull/191978","mergeCommit":{"message":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)\n\n**Resolves: https://github.com/elastic/kibana/issues/192256**\r\n\r\n## Summary\r\n\r\nThis PR re-enables two Cypress test files that didn't run on CI:\r\n`update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also\r\nfixes failing tests in `prebuilt_rules_preview.cy.ts`.\r\n\r\n### Changes\r\n- Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run\r\non CI because it wasn't picked up by a glob\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14).\r\n - `prebuilt_rules_preview.cy.ts`:\r\n- Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe`\r\nblock instead of having it in a variable that is used in every\r\n`describe`. Apparently the tool we use to parse tags doesn't recognize\r\ntags in variables anymore, so this test didn't run in either ESS or\r\nServerless pipelines.\r\n- Removed `describe('All environments' ... ` wrappers since they don't\r\nadd any value anymore. Didn't remove any actual tests.\r\n- Reverted a change from this\r\n[PR](#181427) that added a\r\nbackdrop to the modal which doesn't allow user to switch rules without\r\nclosing the modal. We have a\r\n[test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182)\r\nthat checks that such switching is possible and this test started to\r\nfail once I reactivated the test file.\r\n- Fixed selectors that grab filters in the Overview tab. The old ones\r\nstopped working. Probably because of a change to the filters component\r\nthat is built by another team.\r\n\r\n\r\n#### Correct behaviour: Switching between rules with flyout open\r\n\r\nhttps://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798","sha":"c65c2ae4900a9f75db872cee056af35ff5f79cdc"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191978","number":191978,"mergeCommit":{"message":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)\n\n**Resolves: https://github.com/elastic/kibana/issues/192256**\r\n\r\n## Summary\r\n\r\nThis PR re-enables two Cypress test files that didn't run on CI:\r\n`update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also\r\nfixes failing tests in `prebuilt_rules_preview.cy.ts`.\r\n\r\n### Changes\r\n- Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run\r\non CI because it wasn't picked up by a glob\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14).\r\n - `prebuilt_rules_preview.cy.ts`:\r\n- Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe`\r\nblock instead of having it in a variable that is used in every\r\n`describe`. Apparently the tool we use to parse tags doesn't recognize\r\ntags in variables anymore, so this test didn't run in either ESS or\r\nServerless pipelines.\r\n- Removed `describe('All environments' ... ` wrappers since they don't\r\nadd any value anymore. Didn't remove any actual tests.\r\n- Reverted a change from this\r\n[PR](#181427) that added a\r\nbackdrop to the modal which doesn't allow user to switch rules without\r\nclosing the modal. We have a\r\n[test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182)\r\nthat checks that such switching is possible and this test started to\r\nfail once I reactivated the test file.\r\n- Fixed selectors that grab filters in the Overview tab. The old ones\r\nstopped working. Probably because of a change to the filters component\r\nthat is built by another team.\r\n\r\n\r\n#### Correct behaviour: Switching between rules with flyout open\r\n\r\nhttps://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798","sha":"c65c2ae4900a9f75db872cee056af35ff5f79cdc"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Nikita Indik <nikita.indik@elastic.co>
kibanamachine
referenced
this issue
Sep 13, 2024
…unning in CI (#191978) (#192853) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)](#191978) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Nikita Indik","email":"nikita.indik@elastic.co"},"sourceCommit":{"committedDate":"2024-09-13T12:00:57Z","message":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)\n\n**Resolves: https://github.com/elastic/kibana/issues/192256**\r\n\r\n## Summary\r\n\r\nThis PR re-enables two Cypress test files that didn't run on CI:\r\n`update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also\r\nfixes failing tests in `prebuilt_rules_preview.cy.ts`.\r\n\r\n### Changes\r\n- Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run\r\non CI because it wasn't picked up by a glob\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14).\r\n - `prebuilt_rules_preview.cy.ts`:\r\n- Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe`\r\nblock instead of having it in a variable that is used in every\r\n`describe`. Apparently the tool we use to parse tags doesn't recognize\r\ntags in variables anymore, so this test didn't run in either ESS or\r\nServerless pipelines.\r\n- Removed `describe('All environments' ... ` wrappers since they don't\r\nadd any value anymore. Didn't remove any actual tests.\r\n- Reverted a change from this\r\n[PR](#181427) that added a\r\nbackdrop to the modal which doesn't allow user to switch rules without\r\nclosing the modal. We have a\r\n[test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182)\r\nthat checks that such switching is possible and this test started to\r\nfail once I reactivated the test file.\r\n- Fixed selectors that grab filters in the Overview tab. The old ones\r\nstopped working. Probably because of a change to the filters component\r\nthat is built by another team.\r\n\r\n\r\n#### Correct behaviour: Switching between rules with flyout open\r\n\r\nhttps://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798","sha":"c65c2ae4900a9f75db872cee056af35ff5f79cdc","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","backport:prev-major","v8.16.0"],"title":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI","number":191978,"url":"https://github.com/elastic/kibana/pull/191978","mergeCommit":{"message":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)\n\n**Resolves: https://github.com/elastic/kibana/issues/192256**\r\n\r\n## Summary\r\n\r\nThis PR re-enables two Cypress test files that didn't run on CI:\r\n`update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also\r\nfixes failing tests in `prebuilt_rules_preview.cy.ts`.\r\n\r\n### Changes\r\n- Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run\r\non CI because it wasn't picked up by a glob\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14).\r\n - `prebuilt_rules_preview.cy.ts`:\r\n- Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe`\r\nblock instead of having it in a variable that is used in every\r\n`describe`. Apparently the tool we use to parse tags doesn't recognize\r\ntags in variables anymore, so this test didn't run in either ESS or\r\nServerless pipelines.\r\n- Removed `describe('All environments' ... ` wrappers since they don't\r\nadd any value anymore. Didn't remove any actual tests.\r\n- Reverted a change from this\r\n[PR](#181427) that added a\r\nbackdrop to the modal which doesn't allow user to switch rules without\r\nclosing the modal. We have a\r\n[test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182)\r\nthat checks that such switching is possible and this test started to\r\nfail once I reactivated the test file.\r\n- Fixed selectors that grab filters in the Overview tab. The old ones\r\nstopped working. Probably because of a change to the filters component\r\nthat is built by another team.\r\n\r\n\r\n#### Correct behaviour: Switching between rules with flyout open\r\n\r\nhttps://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798","sha":"c65c2ae4900a9f75db872cee056af35ff5f79cdc"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191978","number":191978,"mergeCommit":{"message":"[Security Solution] Fix some Prebuilt Rules Cypress tests not running in CI (#191978)\n\n**Resolves: https://github.com/elastic/kibana/issues/192256**\r\n\r\n## Summary\r\n\r\nThis PR re-enables two Cypress test files that didn't run on CI:\r\n`update_workflow.cy.ts` and `prebuilt_rules_preview.cy.ts`. It also\r\nfixes failing tests in `prebuilt_rules_preview.cy.ts`.\r\n\r\n### Changes\r\n- Renamed `update_workflow.ts` -> `update_workflow.cy.ts`. It didn't run\r\non CI because it wasn't picked up by a glob\r\n[here](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/package.json#L14).\r\n - `prebuilt_rules_preview.cy.ts`:\r\n- Moved `{ tags: ['@ess', '@serverless'] }` to the top-level `describe`\r\nblock instead of having it in a variable that is used in every\r\n`describe`. Apparently the tool we use to parse tags doesn't recognize\r\ntags in variables anymore, so this test didn't run in either ESS or\r\nServerless pipelines.\r\n- Removed `describe('All environments' ... ` wrappers since they don't\r\nadd any value anymore. Didn't remove any actual tests.\r\n- Reverted a change from this\r\n[PR](#181427) that added a\r\nbackdrop to the modal which doesn't allow user to switch rules without\r\nclosing the modal. We have a\r\n[test](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/prebuilt_rules_preview.cy.ts#L1182)\r\nthat checks that such switching is possible and this test started to\r\nfail once I reactivated the test file.\r\n- Fixed selectors that grab filters in the Overview tab. The old ones\r\nstopped working. Probably because of a change to the filters component\r\nthat is built by another team.\r\n\r\n\r\n#### Correct behaviour: Switching between rules with flyout open\r\n\r\nhttps://github.com/user-attachments/assets/da4a0902-657c-45fe-adc1-eb44ad0de798","sha":"c65c2ae4900a9f75db872cee056af35ff5f79cdc"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Nikita Indik <nikita.indik@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
PR: #191978
Summary
Two Cypress test files are not getting executed on CI.
update_workflow.ts- has incorrect file extension, should be.cy.tsprebuilt_rules_preview.cy.ts- has@essand@serverlesslabels defined in a JS variable, which doesn't work with our tags parser.We need to fix the two issues above to make these tests run again.
The text was updated successfully, but these errors were encountered: