Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

Merged

Conversation

paul-tavares
Copy link
Contributor

Summary

Re-introduces commit 57f59bd which was backed out due to a bug

Original Commit: #100727

  • new data generator for both Isolate and UnIsolate Fleet actions
  • Endpoint test/dev data loader was enhanced to load actions for each endpoint (aka: agent) when run with the --fleet option

Closes #100800

paul-tavares and others added 2 commits May 27, 2021 14:11
…endpoint generator and test data loader (elastic#100727)

* Generate random isolation values for endpoint metadata
* Generator for Fleet Actions
* Added creation of actions to the index test data loader

(cherry picked from commit 57f59bd)
@paul-tavares paul-tavares added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v7.14.0 auto-backport Deprecated - use backport:version if exact versions are needed labels May 27, 2021
@paul-tavares paul-tavares self-assigned this May 27, 2021
@paul-tavares paul-tavares requested a review from a team as a code owner May 27, 2021 18:43
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)


/** Generate either `true` or `false` */
protected randomBoolean(): boolean {
return this.random() < 0.5;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the bug fix for the problem that was introduced originally is here. Instead of using Math.random(), we need to use this.random() which is a seeded random number generator

@paul-tavares
Copy link
Contributor Author

@oatkiller , @jonathan-buttner ,
Pinging both of you based on who last touched the resolver test file x-pack/plugins/security_solution/public/resolver/models/indexed_process_tree/isometric_taxi_layout.test.ts. I had to update the snapshot because the generation of the Endpoint Host metadata is making an additional call to the seeded random number generator (this.random() which used seedrandom package). Because of that additional call, some of the resolver tests failed because the order changed.

In talking with @pzl , having the data generator tied/coupled to tests seems troublesome. We might break out the generation of the Endpoint Host metadata into a separate generator (similar to the new one under x-pack/plugins/security_solution/common/endpoint/data_generators/**) so that we don't impact these test cases going forward.

@jonathan-buttner
Copy link
Contributor

@oatkiller , @jonathan-buttner ,
Pinging both of you based on who last touched the resolver test file x-pack/plugins/security_solution/public/resolver/models/indexed_process_tree/isometric_taxi_layout.test.ts. I had to update the snapshot because the generation of the Endpoint Host metadata is making an additional call to the seeded random number generator (this.random() which used seedrandom package). Because of that additional call, some of the resolver tests failed because the order changed.

In talking with @pzl , having the data generator tied/coupled to tests seems troublesome. We might break out the generation of the Endpoint Host metadata into a separate generator (similar to the new one under x-pack/plugins/security_solution/common/endpoint/data_generators/**) so that we don't impact these test cases going forward.

Yeah that might have been a poor choice on my part 😬 . Maybe we should hard code the values in the tests. It was just really convenient to have the generator create the data that we needed at the time.

@paul-tavares
Copy link
Contributor Author

@jonathan-buttner thanks for the review. No worries. Yeah, we have used the endpoint generator in our tests, but usually will statically set the properties we're looking to test. we also try not to use snapshots too much because of issues like this 😄

Thanks again

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / "before all" hook for "should not allow user with read only privileges to attach alerts to cases".Alerts timeline Privileges: read only "before all" hook for "should not allow user with read only privileges to attach alerts to cases"

Link to Jenkins

Stack Trace

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

AssertionError: Timed out retrying after 60000ms: Expected <span.euiLoadingContent> not to exist in the DOM, but it was continuously found.

Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Alerts timeline`

Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail
    at Object.waitForAlertsPanelToBeLoaded (http://localhost:61161/__cypress/tests?p=cypress/integration/detection_alerts/attach_to_case.spec.ts:19450:43)
    at Context.eval (http://localhost:61161/__cypress/tests?p=cypress/integration/detection_alerts/attach_to_case.spec.ts:18539:18)

Metrics [docs]

Unknown metric groups

References to deprecated APIs

id before after diff
fleet 22 20 -2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @paul-tavares

@paul-tavares paul-tavares merged commit e3517ed into elastic:master May 28, 2021
@paul-tavares paul-tavares deleted the fix/kibana-100800-generator-changes branch May 28, 2021 14:40
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request May 28, 2021
…ort for Host Isolation (elastic#100813)

Re-introduces the changes from elastic#100727 which was backed out due to a bug. Changes included:

* Generate random isolation values for endpoint metadata
* Generator for Fleet Actions
* Added creation of actions to the index test data loader

Plus:

* Fix generator `randomBoolean()` to ensure it works with seeded random numbers
* Update resolver snapshots due to additional call to randomizer
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request May 28, 2021
…ort for Host Isolation (#100813) (#100904)

Re-introduces the changes from #100727 which was backed out due to a bug. Changes included:

* Generate random isolation values for endpoint metadata
* Generator for Fleet Actions
* Added creation of actions to the index test data loader

Plus:

* Fix generator `randomBoolean()` to ensure it works with seeded random numbers
* Update resolver snapshots due to additional call to randomizer

Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 28, 2021
* master: (77 commits)
  [RAC][Security Solution] Register Security Detection Rules with Rule Registry (elastic#96015)
  [Enterprise Search] Log warning for Kibana/EntSearch version mismatches (elastic#100809)
  updating the saved objects test to include more saved object types (elastic#100828)
  [ML] Fix categorization job view examples link when datafeed uses multiple indices (elastic#100789)
  Fixing ES archive mapping failure (elastic#100835)
  Fix bug with Observability > APM header navigation (elastic#100845)
  [Security Solution][Endpoint] Add event filters summary card to the fleet endpoint tab (elastic#100668)
  [Actions] Taking space id into account when creating email footer link (elastic#100734)
  Ensure comments on parameters in arrow functions are captured in the docs and ci metrics. (elastic#100823)
  [Security Solution] Improve find rule and find rule status route performance (elastic#99678)
  [DOCS] Adds video to introduction (elastic#100906)
  [Fleet] Improve combo box for fleet settings (elastic#100603)
  [Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation (elastic#100813)
  [DOCS] Adds Lens video (elastic#100898)
  [TSVB] [Table tab] Fix "Math" aggregation (elastic#100765)
  chore(NA): moving @kbn/io-ts-utils into bazel (elastic#100810)
  [Alerting] Adding feature flag for enabling/disabling rule import and export (elastic#100718)
  [TSVB] Fix Upgrading from 7.12.1 to 7.13.0 breaks TSVB (elastic#100864)
  [Lens] Adds dynamic table cell coloring (elastic#95217)
  [Security Solution][Endpoint] Do not display searchbar in security-trusted apps if there are no items (elastic#100853)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v7.14.0 v8.0.0
Projects
None yet
5 participants