Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Platform] - Fixing exceptions export format #114920

Merged
merged 3 commits into from
Oct 14, 2021

Conversation

yctercero
Copy link
Contributor

@yctercero yctercero commented Oct 13, 2021

Summary

Fixes exceptions export route. It is formatting exports weirdly with extra charachters. Adds integration tests for export of exception list.

Existing export
"{\"_version\":\"WzIxMjU1LDNd\",\"created_at\":\"2021-10-13T03:35:46.040Z\",\"created_by\":\"ytercero\",\"description\":\"test new\",\"id\":\"a6a3ef80-2bd6-11ec-8555-95b02ad6ae04\",\"immutable\":false,\"list_id\":\"15007788-8227-4b29-bec0-417cb9bd2192\",\"name\":\"test new\",\"namespace_type\":\"single\",\"os_types\":[],\"tags\":[],\"tie_breaker_id\":\"2b368e6a-f469-4a3b-8bca-6dd4bf8b82c9\",\"type\":\"detection\",\"updated_at\":\"2021-10-13T03:35:46.110Z\",\"updated_by\":\"ytercero\",\"version\":1}\n" "{\"_version\":\"WzIxMjU5LDNd\",\"comments\":[],\"created_at\":\"2021-10-13T03:36:01.327Z\",\"created_by\":\"ytercero\",\"description\":\"test new - exception list item\",\"entries\":[{\"field\":\"host.name\",\"operator\":\"included\",\"type\":\"match\",\"value\":\"DESKTOP-QBBSCUT\"}],\"id\":\"afc08bf0-2bd6-11ec-8555-95b02ad6ae04\",\"item_id\":\"f0dd9782-5a5a-43f9-ab9c-cb8c2b6fc4e5\",\"list_id\":\"15007788-8227-4b29-bec0-417cb9bd2192\",\"name\":\"test new - exception list item\",\"namespace_type\":\"single\",\"os_types\":[],\"tags\":[],\"tie_breaker_id\":\"d6eb9a6d-e3e1-4038-b476-f17cf3902e16\",\"type\":\"simple\",\"updated_at\":\"2021-10-13T03:36:01.396Z\",\"updated_by\":\"ytercero\"}\n" {"exception_list_items_details":"{\"exported_count\":1}\n"}
Fixed export
{"_version":"WzU0MjIzLDNd","created_at":"2021-10-13T03:35:46.040Z","created_by":"ytercero","description":"test new","id":"a6a3ef80-2bd6-11ec-8555-95b02ad6ae04","immutable":false,"list_id":"15007788-8227-4b29-bec0-417cb9bd2192","name":"test new","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"2b368e6a-f469-4a3b-8bca-6dd4bf8b82c9","type":"detection","updated_at":"2021-10-13T03:35:46.110Z","updated_by":"ytercero","version":1} {"_version":"WzU0MjI1LDNd","comments":[],"created_at":"2021-10-13T03:36:01.327Z","created_by":"ytercero","description":"test new - exception list item","entries":[{"field":"host.name","operator":"included","type":"match","value":"DESKTOP-QBBSCUT"}],"id":"afc08bf0-2bd6-11ec-8555-95b02ad6ae04","item_id":"f0dd9782-5a5a-43f9-ab9c-cb8c2b6fc4e5","list_id":"15007788-8227-4b29-bec0-417cb9bd2192","name":"test new - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"d6eb9a6d-e3e1-4038-b476-f17cf3902e16","type":"simple","updated_at":"2021-10-13T03:36:01.396Z","updated_by":"ytercero"} {"exported_list_items_count":1}

Testing

To test, create a rule with and add an exception item to your rule. Navigate to the exceptions tab and hit export:
Screen Shot 2021-10-13 at 2 03 40 PM

Checklist

For maintainers

Not sure this is a breaking change, but happy to add tag if needed.

@yctercero yctercero added bug Fixes for quality problems that affect the customer experience release_note:fix Team:Security Solution Platform Security Solution Platform Team v7.16.0 v8.0.0 labels Oct 13, 2021
@yctercero yctercero self-assigned this Oct 13, 2021
* Side Public License, v 1.
*/

export const transformDataToNdjson = (data: unknown[]): string => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this to package as it's used in multiple plugins and exactly the same.

@yctercero yctercero marked this pull request as ready for review October 13, 2021 21:08
@yctercero yctercero requested review from a team as code owners October 13, 2021 21:08
Copy link
Contributor

@rylnd rylnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Integration tests look great 👍

@yctercero yctercero enabled auto-merge (squash) October 14, 2021 01:31
@yctercero yctercero added the auto-backport Deprecated - use backport:version if exact versions are needed label Oct 14, 2021
@yctercero yctercero changed the title Fixing exceptions export format [Security Solution][Platform] - Fixing exceptions export format Oct 14, 2021
@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/search/session·ts.apis search search session touched time updates when you poll on an search

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]     │
[00:00:00]       └-: apis
[00:00:00]         └-> "before all" hook in "apis"
[00:00:00]         └-: search
[00:00:00]           └-> "before all" hook in "search"
[00:00:03]           └-: search session
[00:00:03]             └-> "before all" hook for "should fail to extend a nonexistent session"
[00:00:03]             └-> should fail to extend a nonexistent session
[00:00:03]               └-> "before each" hook: global before each for "should fail to extend a nonexistent session"
[00:00:03]               │ proc [kibana] [2021-10-14T02:08:51.554+00:00][ERROR][plugins.dataEnhanced.data_enhanced] [object Object]
[00:00:03]               └- ✓ pass  (71ms) "apis search search session should fail to extend a nonexistent session"
[00:00:03]             └-> should sync search ids into not persisted session
[00:00:03]               └-> "before each" hook: global before each for "should sync search ids into not persisted session"
[00:00:03]               │ debg Waiting up to 5000ms for searches persisted into session...
[00:00:03]               │ proc [kibana] [2021-10-14T02:08:51.689+00:00][ERROR][plugins.dataEnhanced.data_enhanced] [object Object]
[00:00:03]               │ debg --- retry.waitForWithTimeout error: expected 200 "OK", got 404 "Not Found"
[00:00:04]               │ proc [kibana] [2021-10-14T02:08:52.270+00:00][ERROR][plugins.dataEnhanced.data_enhanced] [object Object]
[00:00:04]               │ debg --- retry.waitForWithTimeout failed again with the same message...
[00:00:04]               │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/-bVUJorbR_6ql_d2YB5rPQ] update_mapping [_doc]
[00:00:04]               └- ✓ pass  (1.2s) "apis search search session should sync search ids into not persisted session"
[00:00:04]             └-> should complete session when searches complete
[00:00:04]               └-> "before each" hook: global before each for "should complete session when searches complete"
[00:00:05]               │ debg Waiting up to 5000ms for searches persisted into session...
[00:00:05]               │ debg --- retry.waitForWithTimeout error: expected [] to contain 'FnZaeFlkVEN0VE42SWlsQ2FGWUtQTVEbRmNVTjBlX2JRb0tiUWdnMUd2RjhIZzoyMjQx'
[00:00:05]               │ debg --- retry.waitForWithTimeout failed again with the same message...
[00:00:08]               │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/-bVUJorbR_6ql_d2YB5rPQ] update_mapping [_doc]
[00:00:16]               │ debg Waiting up to 5000ms for searches eventually complete and session gets into the complete state...
[00:00:16]               └- ✓ pass  (11.4s) "apis search search session should complete session when searches complete"
[00:00:16]             └-> touched time updates when you poll on an search
[00:00:16]               └-> "before each" hook: global before each for "touched time updates when you poll on an search"
[00:00:16]               │ debg Waiting up to 20000ms for search session created...
[00:00:16]               │ proc [kibana] [2021-10-14T02:09:04.308+00:00][ERROR][plugins.dataEnhanced.data_enhanced] [object Object]
[00:00:16]               │ proc [kibana] [2021-10-14T02:09:04.877+00:00][ERROR][plugins.dataEnhanced.data_enhanced] [object Object]
[00:00:20]               └- ✖ fail: apis search search session touched time updates when you poll on an search
[00:00:20]               │      Error: expected '2021-10-14T02:09:05.295Z' to be below 2021-10-14T02:09:05.295Z
[00:00:20]               │       at Assertion.assert (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:00:20]               │       at Assertion.lessThan.Assertion.below (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/expect/expect.js:336:8)
[00:00:20]               │       at Function.lessThan (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/expect/expect.js:531:15)
[00:00:20]               │       at Context.<anonymous> (test/api_integration/apis/search/session.ts:438:65)
[00:00:20]               │       at runMicrotasks (<anonymous>)
[00:00:20]               │       at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:00:20]               │       at Object.apply (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:00:20]               │ 
[00:00:20]               │ 

Stack Trace

Error: expected '2021-10-14T02:09:05.295Z' to be below 2021-10-14T02:09:05.295Z
    at Assertion.assert (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.lessThan.Assertion.below (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/expect/expect.js:336:8)
    at Function.lessThan (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/expect/expect.js:531:15)
    at Context.<anonymous> (test/api_integration/apis/search/session.ts:438:65)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at Object.apply (/dev/shm/workspace/parallel/21/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
lists 320 321 +1
securitySolution 2749 2750 +1
total +2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/securitysolution-utils 2 4 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
lists 148.2KB 148.5KB +397.0B
securitySolution 4.6MB 4.6MB +406.0B
total +803.0B
Unknown metric groups

API count

id before after diff
@kbn/securitysolution-utils 4 6 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @yctercero

@yctercero yctercero merged commit 69a6cf3 into elastic:master Oct 14, 2021
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 14, 2021
### Summary

Fixing exceptions export format and adding integration tests for it.
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Oct 14, 2021
### Summary

Fixing exceptions export format and adding integration tests for it.

Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
jloleysens added a commit to jloleysens/kibana that referenced this pull request Oct 14, 2021
…mple/introduce-baseline-tests

* 'master' of github.com:elastic/kibana: (55 commits)
  [Fleet] Improve Functionality around Managed Package Policies (elastic#114526)
  cleanup (elastic#114902)
  remove stray semicolon (elastic#114969)
  [Security Solution] Edit host isolation exception IP UI (elastic#114279)
  [ML] APM Correlations: Round duration values to be used in range aggregations. (elastic#114833)
  [Index Management] Added `data-test-subj` values to the index context menu buttons (elastic#114900)
  [Stack monitoring] Fix logstash functional tests for react (elastic#114819)
  Implement hybrid approach to writing rule execution event logs (elastic#114852)
  [Detection Rules] Add 7.16 rules (elastic#114939)
  Fixing exceptions export format (elastic#114920)
  Clean up inaccurate comments (elastic#114935)
  chore(NA): fixes a typo on persist_bazel_cache.sh comment (elastic#114943)
  [ci] Fixes Bazel cache writes (elastic#114915)
  fix package.json: (elastic#114936)
  [Controls] Redux Toolkit and Embeddable Redux Wrapper (elastic#114371)
  [APM] Fixes incorrect index config names (elastic#114901) (elastic#114904)
  [Workplace Search] Fix button order and remove extra source name label (elastic#114899)
  [Actions] Fixed actions telemetry for multiple namespaces usage (elastic#114748)
  docs: fix config names (elastic#114903)
  Update kibana to EMS 7.16 (elastic#114865)
  ...
@KOTungseth KOTungseth added the Feature:Detection Alerts Security Solution Detection Alerts Feature label Nov 19, 2021
@yctercero yctercero deleted the exceptions_export branch August 4, 2022 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed bug Fixes for quality problems that affect the customer experience Feature:Detection Alerts Security Solution Detection Alerts Feature release_note:fix Team:Security Solution Platform Security Solution Platform Team v7.16.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants