Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Cloud Security] Update CSP Version to 1.9.0 for Test #186657

Merged
merged 14 commits into from
Jul 2, 2024
Merged

[Cloud Security] Update CSP Version to 1.9.0 for Test #186657

merged 14 commits into from
Jul 2, 2024

Conversation

animehart
Copy link
Contributor

Summary

With 8.14 released, we want to make sure our CSP is using the latest CSP version for our test environment

@animehart animehart changed the title update test version [Cloud Security] Update CSP Version to 1.9.0 for Test Jun 21, 2024
@animehart
Copy link
Contributor Author

/ci

@animehart
Copy link
Contributor Author

/ci

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6366

[❌] x-pack/test/cloud_security_posture_functional/config.ts: 0/50 tests passed.
[✅] x-pack/test/cloud_security_posture_api/config.ts: 50/50 tests passed.
[✅] x-pack/test_serverless/functional/test_suites/security/config.cloud_security_posture.essentials.ts: 50/50 tests passed.
[✅] x-pack/test_serverless/functional/test_suites/security/config.cloud_security_posture.basic.ts: 50/50 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6367

[❌] x-pack/test/cloud_security_posture_functional/config.ts: 0/50 tests passed.
[✅] x-pack/test/cloud_security_posture_api/config.ts: 50/50 tests passed.
[✅] x-pack/test_serverless/functional/test_suites/security/config.cloud_security_posture.essentials.ts: 50/50 tests passed.
[✅] x-pack/test_serverless/functional/test_suites/security/config.cloud_security_posture.basic.ts: 50/50 tests passed.

see run history

@animehart
Copy link
Contributor Author

/ci

@animehart
Copy link
Contributor Author

/ci

@animehart
Copy link
Contributor Author

/ci

@animehart animehart marked this pull request as ready for review June 22, 2024 06:01
@animehart animehart requested a review from a team as a code owner June 22, 2024 06:01
@animehart animehart added v8.15.0 Team:Cloud Security Cloud Security team related labels Jun 22, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@animehart animehart added backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes labels Jun 22, 2024
@elastic elastic deleted a comment from kibanamachine Jun 22, 2024
maxcold
maxcold requested changes Jun 24, 2024
View reviewed changes
Copy link
Contributor

@maxcold maxcold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest finding a way to make tests less brittle

...ck/test/cloud_security_posture_functional/pages/cis_integrations/cspm/cis_integration_aws.ts Outdated
@@ -124,6 +124,7 @@ export default function (providerContext: FtrProviderContext) {
(await cisIntegration.getFieldValueInEditPage(DIRECT_ACCESS_KEY_ID_TEST_ID)) ===
directAccessKeyId
).to.be(true);
expect(await cisIntegration.doesStringExist('Replace secret access key')).to.not.be(null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is very brittle to rely on specific text and on the generic markup (span>text). Can we add a proper test id to this component and rely on it? Otherwise every time the wording change in fleet our tests will break

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test ID to query for the element?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this component is actually a fleet component that I use Lazy import to use in our plugin
the reason why I'm using text is because I thought it might be better to not touch their component

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a problem in adding a qa attribute to the component in Fleet, any specific concern you have @animehart ? The fact that another team owns the component makes it even more important - we don't want our tests to break because of the wording changes made by another team. The Fleet team also won't be happy to update our tests when they change smth

seanrathier
seanrathier reviewed Jun 24, 2024
View reviewed changes
Copy link
Contributor

@seanrathier seanrathier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to @maxcold comment. Should we consider moving this to an integration test with MSW?

...ck/test/cloud_security_posture_functional/pages/cis_integrations/cspm/cis_integration_aws.ts Outdated
@@ -124,6 +124,7 @@ export default function (providerContext: FtrProviderContext) {
(await cisIntegration.getFieldValueInEditPage(DIRECT_ACCESS_KEY_ID_TEST_ID)) ===
directAccessKeyId
).to.be(true);
expect(await cisIntegration.doesStringExist('Replace secret access key')).to.not.be(null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test ID to query for the element?

@animehart animehart requested a review from a team as a code owner June 24, 2024 19:08
@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Jun 24, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] x-pack/test/cases_api_integration/security_and_spaces/config_trial.ts / cases security and spaces enabled: trial get_connectors push needsToBePushed sets needs to push to false the service now connector and true for jira

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 1.4MB 1.4MB +38.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@animehart animehart requested a review from maxcold June 26, 2024 19:33
maxcold
maxcold approved these changes Jun 27, 2024
View reviewed changes
Copy link
Contributor

@maxcold maxcold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, one suggestion on how to construct the test subject a bit more consistently

...cy/create_package_policy_page/components/steps/components/package_policy_input_var_field.tsx Outdated Show resolved Hide resolved
nchaulet
nchaulet approved these changes Jul 2, 2024
View reviewed changes
Copy link
Member

@nchaulet nchaulet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fleet changes LGTM 🚀

@animehart animehart merged commit 9fae9c5 into elastic:main Jul 2, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seanrathier seanrathier seanrathier left review comments

@nchaulet nchaulet nchaulet approved these changes

@maxcold maxcold maxcold approved these changes

Assignees
No one assigned
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related Team:Fleet Team label for Observability Data Collection Fleet team v8.15.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@animehart @kibanamachine @elasticmachine @kibana-ci @maxcold @nchaulet @seanrathier