Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: X-Pack Detection Engine API Integration Tests.x-pack/test/detection_engine_api_integration/security_and_spaces/tests/create_threat_matching·ts - detection engine api security and spaces enabled create_threat_matching tests with auditbeat data indicator enrichment generates multiple signals with multiple matches #93152

Closed
kibanamachine opened this issue Mar 2, 2021 · 21 comments
Closed

Failing test: X-Pack Detection Engine API Integration Tests.x-pack/test/detection_engine_api_integration/security_and_spaces/tests/create_threat_matching·ts - detection engine api security and spaces enabled create_threat_matching tests with auditbeat data indicator enrichment generates multiple signals with multiple matches #93152

kibanamachine opened this issue Mar 2, 2021 · 21 comments
Assignees
@rylnd
Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test skipped-test Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.0.0

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Mar 2, 2021 •
edited
Loading

A test failed on a tracked branch

Error: expected [ { indicator: [ [Object] ] },
  { indicator: [ [Object], [Object], [Object] ] } ] to sort of equal [ { indicator: [ [Object] ] },
  { indicator: [ [Object], [Object], [Object] ] } ]
    at Assertion.assert (/dev/shm/workspace/parallel/23/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/23/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.<anonymous> (test/detection_engine_api_integration/security_and_spaces/tests/create_threat_matching.ts:578:30)
    at Object.apply (/dev/shm/workspace/parallel/23/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16) {
  actual: '[\n' +
    '  {\n' +
    '    "indicator": [\n' +
    '      {\n' +
    `        "description": "domain should match the auditbeat hosts' data's source.ip"\n` +
    '        "domain": "159.89.119.67"\n' +
    '        "first_seen": "2021-01-26T11:09:04.000Z"\n' +
    '        "matched": {\n' +
    '          "atomic": "159.89.119.67"\n' +
    '          "field": "destination.ip"\n' +
    '          "id": "978783"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '        "url": {\n' +
    '          "full": "http://159.89.119.67:59600/bin.sh"\n' +
    '          "scheme": "http"\n' +
    '        }\n' +
    '      }\n' +
    '    ]\n' +
    '  }\n' +
    '  {\n' +
    '    "indicator": [\n' +
    '      {\n' +
    '        "description": "this should match auditbeat/hosts on both port and ip"\n' +
    '        "first_seen": "2021-01-26T11:06:03.000Z"\n' +
    '        "ip": "45.115.45.3"\n' +
    '        "matched": {\n' +
    '          "atomic": "45.115.45.3"\n' +
    '          "field": "source.ip"\n' +
    '          "id": "978785"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "port": 57324\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '      }\n' +
    '      {\n' +
    `        "description": "domain should match the auditbeat hosts' data's source.ip"\n` +
    '        "domain": "159.89.119.67"\n' +
    '        "first_seen": "2021-01-26T11:09:04.000Z"\n' +
    '        "matched": {\n' +
    '          "atomic": "159.89.119.67"\n' +
    '          "field": "destination.ip"\n' +
    '          "id": "978783"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '        "url": {\n' +
    '          "full": "http://159.89.119.67:59600/bin.sh"\n' +
    '          "scheme": "http"\n' +
    '        }\n' +
    '      }\n' +
    '      {\n' +
    '        "description": "this should match auditbeat/hosts on both port and ip"\n' +
    '        "first_seen": "2021-01-26T11:06:03.000Z"\n' +
    '        "ip": "45.115.45.3"\n' +
    '        "matched": {\n' +
    '          "atomic": 57324\n' +
    '          "field": "source.port"\n' +
    '          "id": "978785"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "port": 57324\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '      }\n' +
    '    ]\n' +
    '  }\n' +
    ']',
  expected: '[\n' +
    '  {\n' +
    '    "indicator": [\n' +
    '      {\n' +
    `        "description": "domain should match the auditbeat hosts' data's source.ip"\n` +
    '        "domain": "159.89.119.67"\n' +
    '        "first_seen": "2021-01-26T11:09:04.000Z"\n' +
    '        "matched": {\n' +
    '          "atomic": "159.89.119.67"\n' +
    '          "field": "destination.ip"\n' +
    '          "id": "978783"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '        "url": {\n' +
    '          "full": "http://159.89.119.67:59600/bin.sh"\n' +
    '          "scheme": "http"\n' +
    '        }\n' +
    '      }\n' +
    '    ]\n' +
    '  }\n' +
    '  {\n' +
    '    "indicator": [\n' +
    '      {\n' +
    `        "description": "domain should match the auditbeat hosts' data's source.ip"\n` +
    '        "domain": "159.89.119.67"\n' +
    '        "first_seen": "2021-01-26T11:09:04.000Z"\n' +
    '        "matched": {\n' +
    '          "atomic": "159.89.119.67"\n' +
    '          "field": "destination.ip"\n' +
    '          "id": "978783"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '        "url": {\n' +
    '          "full": "http://159.89.119.67:59600/bin.sh"\n' +
    '          "scheme": "http"\n' +
    '        }\n' +
    '      }\n' +
    '      {\n' +
    '        "description": "this should match auditbeat/hosts on both port and ip"\n' +
    '        "first_seen": "2021-01-26T11:06:03.000Z"\n' +
    '        "ip": "45.115.45.3"\n' +
    '        "matched": {\n' +
    '          "atomic": "45.115.45.3"\n' +
    '          "field": "source.ip"\n' +
    '          "id": "978785"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "port": 57324\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '      }\n' +
    '      {\n' +
    '        "description": "this should match auditbeat/hosts on both port and ip"\n' +
    '        "first_seen": "2021-01-26T11:06:03.000Z"\n' +
    '        "ip": "45.115.45.3"\n' +
    '        "matched": {\n' +
    '          "atomic": 57324\n' +
    '          "field": "source.port"\n' +
    '          "id": "978785"\n' +
    '          "index": "filebeat-8.0.0-2021.01.26-000001"\n' +
    '          "type": "url"\n' +
    '        }\n' +
    '        "port": 57324\n' +
    '        "provider": "geenensp"\n' +
    '        "type": "url"\n' +
    '      }\n' +
    '    ]\n' +
    '  }\n' +
    ']',
  showDiff: true
}

First failure: Jenkins Build
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Mar 2, 2021
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

mistic added a commit that referenced this issue Mar 2, 2021
@mistic
skip flaky suite (#93152)
a8adbbf
mistic added a commit that referenced this issue Mar 2, 2021
@mistic
skip flaky suite (#93152)
91168ee
@mistic mistic added the Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! label Mar 2, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-security (Team:Security)

@mistic
Copy link
Member

mistic commented Mar 2, 2021 •
edited
Loading

This test has been failing and looks flaky. Skipping for now.

master/8.0: b33ea36
7.x/7.13: 9eb6fb7

@legrego
Copy link
Member

legrego commented Mar 2, 2021

@mistic this test belongs to the security solutions team, not the security team. I'll reassign for triage

@legrego legrego added Team:Detections and Resp Security Detection Response Team and removed Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! labels Mar 2, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

mistic added a commit that referenced this issue Mar 2, 2021
@mistic
skip flaky suite (#93152)
b33ea36
mistic added a commit that referenced this issue Mar 2, 2021
@mistic
skip flaky suite (#93152)
9eb6fb7
@mistic mistic added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Mar 2, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

gmmorris added a commit to gmmorris/kibana that referenced this issue Mar 2, 2021
@gmmorris
Merge branch 'task-manager/docs-monitoring' of github.com:gmmorris/ki…
0a85947 
…bana into task-manager/docs-monitoring

* 'task-manager/docs-monitoring' of github.com:gmmorris/kibana:
  [ILM] Allow multiple searchable snapshot actions (elastic#92789)
  Improve consistency for display of management items (elastic#92694)
  skip flaky suite (elastic#93152)
  skip flaky suite (elastic#93152)
  [ILM] Refactor edit_policy client integration tests into separate feature files (elastic#92826)
  Add developer documentation about the building blocks we offer plugin developers (elastic#92743)
  [Security Solution] Case ui enhancement (elastic#91863)
  [Security Solution] [Detections] Updates warning message when no indices match provided index patterns (elastic#93094)
  Collect agent telemetry even when fleet server is disabled. (elastic#93198)
  [Lens] Fix runtime validation error message (elastic#93195)
  [Lens] Remove warning about ordinal x-domain (elastic#93049)
  [Security Solution] Fixes the Customize Event Renderers modal by removing the EuiOverlayMask (elastic#93150)
  Cleanup Security plugin imports (elastic#93056)
  [Security Solution] - Bug fixes (elastic#92294)
  Updated doc links (elastic#92968)
  [ML] Transforms: Fixes chart histograms for runtime fields. (elastic#93028)
  [chore] Enable core's eslint rule: `@ts-expect-error` (elastic#93086)
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

gmmorris added a commit to gmmorris/kibana that referenced this issue Mar 2, 2021
@gmmorris
Merge branch 'master' into actions/terminology-api
88098e3 
* master: (199 commits)
  Convert Canvas docs to MDX for use in Elastic Docs (elastic#91969)
  [Bazel] More resilient Workspace Status (elastic#93244)
  [Discover] Change icon of saved search in open search panel and embeddable selection (elastic#93001)
  [Workplace Search] Role Mappings to Kibana (elastic#93123)
  [Fleet] Use type-only imports where possible (elastic#92979)
  [Lens] Set pie chart slices sorted clockwise (elastic#92617)
  Remove ms label from CPU load on status page (elastic#92836)
  [App Search] Migrate Create Meta Engine View (elastic#92127)
  [Time to Visualize] Disable Visualize URL Tracker When Linked to OriginatingApp (elastic#92917)
  [ILM] Allow multiple searchable snapshot actions (elastic#92789)
  Improve consistency for display of management items (elastic#92694)
  skip flaky suite (elastic#93152)
  skip flaky suite (elastic#93152)
  [ILM] Refactor edit_policy client integration tests into separate feature files (elastic#92826)
  Add developer documentation about the building blocks we offer plugin developers (elastic#92743)
  [Security Solution] Case ui enhancement (elastic#91863)
  [Security Solution] [Detections] Updates warning message when no indices match provided index patterns (elastic#93094)
  Collect agent telemetry even when fleet server is disabled. (elastic#93198)
  [Lens] Fix runtime validation error message (elastic#93195)
  [Lens] Remove warning about ordinal x-domain (elastic#93049)
  ...
@rylnd rylnd mentioned this issue Mar 2, 2021
Merged
2 tasks
@rylnd rylnd self-assigned this Mar 2, 2021
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@spalger
Copy link
Contributor

spalger commented Mar 3, 2021

Attempted to fix in #93350, but that was reverted to reskip the test

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

1 similar comment
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@rylnd rylnd mentioned this issue Mar 9, 2021
Merged
2 tasks
@rylnd
Copy link
Contributor

rylnd commented Mar 25, 2021

Looks like this was (actually) closed by #94241. Backported to 7.x and 7.12 as well.

@rylnd rylnd closed this as completed Mar 25, 2021
@kibanamachine kibanamachine reopened this Oct 25, 2021
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 7.16

@mistic mistic closed this as completed Oct 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rylnd rylnd

Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test skipped-test Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.0.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@rylnd @spalger @legrego @mistic @elasticmachine @kibanamachine