Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Event Correlation][Sequence]Investigate in timeline returning no result #120898

Closed
ghost opened this issue Dec 9, 2021 · 21 comments · Fixed by #123333
Closed
Assignees
Labels
bug Fixes for quality problems that affect the customer experience impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. QA:Validated Issue has been validated by QA Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting:Investigations Security Solution Investigations Team v8.0.0

Comments

@ghost
Copy link

ghost commented Dec 9, 2021

Describe the bug
[Event Correlation][Sequence]Investigate in timeline returning no result

Build Details

Version: 8.0.0-SNAPSHOT
Commit: ad3660f3acbfe6eb809d869b908221edf2846313
Build: 48594

Steps

  • Create a EQL Rule with Sequence query
    sequence [ process where process.name == "cmd.exe" ] [ process where process.name == "notepad.exe" ]
  • generate alert for above rule
  • investigate in timeline for above generated alert
  • Observed that no result is filtered for EQL Rule Alert

image

Whats Working
Issue is not occuring for single EQL Query

working.mp4

Screen-cast

timeline.mp4
issue-another.mp4

Extra

JSON for Sequence Alert
{
  "_index": ".internal.alerts-security.alerts-default-000001",
  "_id": "8f5924dc00126cc754e0ea8f19afa089adb3c2365287af1c137474f94e0bc1e2",
  "_score": 1,
  "_source": {
    "kibana.version": "8.0.0-SNAPSHOT",
    "kibana.alert.rule.category": "Event Correlation Rule",
    "kibana.alert.rule.consumer": "siem",
    "kibana.alert.rule.name": "EQL rule",
    "kibana.alert.rule.producer": "siem",
    "kibana.alert.rule.rule_type_id": "siem.eqlRule",
    "kibana.alert.rule.uuid": "21535bc0-58ec-11ec-a41a-131fd10250bd",
    "kibana.space_ids": [
      "default"
    ],
    "tags": [
      "qa",
      "__internal_rule_id:89937845-488e-4eda-b9b4-53699356f5e5",
      "__internal_immutable:false"
    ],
    "@timestamp": "2021-12-09T12:32:59.110Z",
    "agent": {
      "id": "26fb7b05-607c-4f0e-8669-2e06890de80d",
      "type": "endpoint",
      "version": "8.0.0-SNAPSHOT"
    },
    "process": {
      "code_signature": {
        "trusted": true,
        "subject_name": "Microsoft Windows",
        "exists": true,
        "status": "trusted"
      }
    },
    "message": "Endpoint process event",
    "ecs": {
      "version": "1.11.0"
    },
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "endpoint.events.process"
    },
    "elastic": {
      "agent": {
        "id": "26fb7b05-607c-4f0e-8669-2e06890de80d"
      }
    },
    "host": {
      "hostname": "DESKTOP-QBBSCUT",
      "os": {
        "Ext": {
          "variant": "Windows 10 Pro"
        },
        "kernel": "1903 (10.0.18362.1256)",
        "name": "Windows",
        "family": "windows",
        "type": "windows",
        "version": "1903 (10.0.18362.1256)",
        "platform": "windows",
        "full": "Windows 10 Pro 1903 (10.0.18362.1256)"
      },
      "name": "DESKTOP-QBBSCUT",
      "id": "4143c277-074e-47a9-b37d-37f94b508705",
      "architecture": "x86_64"
    },
    "user": {
      "domain": "DESKTOP-QBBSCUT",
      "name": "zeus",
      "id": "S-1-5-21-4215045029-3277270250-148079304-1004"
    },
    "event.agent_id_status": "verified",
    "event.kind": "signal",
    "event.module": "endpoint",
    "event.dataset": "endpoint.events.process",
    "kibana.alert.status": "active",
    "kibana.alert.workflow_status": "open",
    "kibana.alert.depth": 2,
    "kibana.alert.rule.created_at": "2021-12-09T12:32:54.537Z",
    "kibana.alert.rule.created_by": "elastic",
    "kibana.alert.rule.enabled": true,
    "kibana.alert.rule.interval": "5s",
    "kibana.alert.rule.updated_at": "2021-12-09T12:32:56.517Z",
    "kibana.alert.rule.updated_by": "elastic",
    "kibana.alert.rule.description": "test",
    "kibana.alert.rule.risk_score": 21,
    "kibana.alert.rule.severity": "low",
    "kibana.alert.rule.license": "",
    "kibana.alert.rule.meta.from": "30000h",
    "kibana.alert.rule.meta.kibana_siem_app_url": "https://snapshot-7d0fa6.kb.us-central1.gcp.qa.cld.elstc.co:9243/app/security",
    "kibana.alert.rule.author": [],
    "kibana.alert.rule.false_positives": [],
    "kibana.alert.rule.from": "now-108000005s",
    "kibana.alert.rule.rule_id": "89937845-488e-4eda-b9b4-53699356f5e5",
    "kibana.alert.rule.max_signals": 100,
    "kibana.alert.rule.risk_score_mapping": [],
    "kibana.alert.rule.severity_mapping": [],
    "kibana.alert.rule.threat": [],
    "kibana.alert.rule.to": "now",
    "kibana.alert.rule.references": [],
    "kibana.alert.rule.version": 1,
    "kibana.alert.rule.exceptions_list": [],
    "kibana.alert.rule.immutable": false,
    "kibana.alert.rule.type": "eql",
    "kibana.alert.rule.language": "eql",
    "kibana.alert.rule.index": [
      "apm-*-transaction*",
      "traces-apm*",
      "auditbeat-*",
      "endgame-*",
      "filebeat-*",
      "logs-*",
      "packetbeat-*",
      "winlogbeat-*"
    ],
    "kibana.alert.rule.query": "sequence [ process where process.name == \"cmd.exe\" ] [ process where process.name == \"notepad.exe\" ]",
    "kibana.alert.rule.filters": [],
    "kibana.alert.original_event.agent_id_status": "verified",
    "kibana.alert.original_event.kind": "event",
    "kibana.alert.original_event.module": "endpoint",
    "kibana.alert.original_event.dataset": "endpoint.events.process",
    "event": {
      "kind": "signal"
    },
    "kibana.alert.ancestors": [
      {
        "id": "lNzVnX0BWpFDgqW3ZMS_",
        "type": "event",
        "index": ".ds-logs-endpoint.events.process-default-2021.12.09-000001",
        "depth": 0
      },
      {
        "id": "eae12e458361a5da7e57509252a0c1e8fce99e364fbd5e85425a57666df4e349",
        "type": "signal",
        "index": "",
        "depth": 1,
        "rule": "21535bc0-58ec-11ec-a41a-131fd10250bd"
      },
      {
        "id": "RV5Enn0BCj1zhlBeakic",
        "type": "event",
        "index": ".ds-logs-endpoint.events.process-default-2021.12.09-000001",
        "depth": 0
      },
      {
        "id": "cd8dcbec32d44a5737d2bd077e28d7df1a6276b61126cb287be7efeecd7a19f0",
        "type": "signal",
        "index": "",
        "depth": 1,
        "rule": "21535bc0-58ec-11ec-a41a-131fd10250bd"
      }
    ],
    "kibana.alert.reason": "event by zeus on DESKTOP-QBBSCUT created low alert EQL rule.",
    "kibana.alert.rule.actions": [],
    "kibana.alert.rule.tags": [
      "qa"
    ],
    "kibana.alert.original_time": "2021-12-09T06:15:36.660Z",
    "kibana.alert.group.id": "8f5924dc00126cc754e0ea8f19afa089adb3c2365287af1c137474f94e0bc1e2"
  },
  "fields": {
    "kibana.alert.rule.updated_by": [
      "elastic"
    ],
    "signal.ancestors.depth": [
      0,
      1,
      0,
      1
    ],
    "host.hostname": [
      "DESKTOP-QBBSCUT"
    ],
    "kibana.alert.rule.tags": [
      "qa"
    ],
    "process.code_signature.exists": [
      true
    ],
    "elastic.agent.id": [
      "26fb7b05-607c-4f0e-8669-2e06890de80d"
    ],
    "signal.rule.enabled": [
      "true"
    ],
    "kibana.alert.ancestors.depth": [
      0,
      1,
      0,
      1
    ],
    "host.os.version": [
      "1903 (10.0.18362.1256)"
    ],
    "signal.rule.max_signals": [
      100
    ],
    "signal.rule.updated_at": [
      "2021-12-09T12:32:56.517Z"
    ],
    "event.agent_id_status": [
      "verified"
    ],
    "kibana.alert.group.id": [
      "8f5924dc00126cc754e0ea8f19afa089adb3c2365287af1c137474f94e0bc1e2"
    ],
    "user.id": [
      "S-1-5-21-4215045029-3277270250-148079304-1004"
    ],
    "host.os.type": [
      "windows"
    ],
    "kibana.alert.original_event.module": [
      "endpoint"
    ],
    "kibana.alert.rule.interval": [
      "5s"
    ],
    "kibana.alert.rule.type": [
      "eql"
    ],
    "tags": [
      "qa",
      "__internal_rule_id:89937845-488e-4eda-b9b4-53699356f5e5",
      "__internal_immutable:false"
    ],
    "host.architecture": [
      "x86_64"
    ],
    "kibana.alert.rule.immutable": [
      "false"
    ],
    "agent.id": [
      "26fb7b05-607c-4f0e-8669-2e06890de80d"
    ],
    "signal.original_event.module": [
      "endpoint"
    ],
    "signal.rule.from": [
      "now-108000005s"
    ],
    "kibana.alert.rule.enabled": [
      "true"
    ],
    "kibana.alert.rule.version": [
      "1"
    ],
    "kibana.alert.ancestors.type": [
      "event",
      "signal",
      "event",
      "signal"
    ],
    "user.name": [
      "zeus"
    ],
    "signal.ancestors.index": [
      ".ds-logs-endpoint.events.process-default-2021.12.09-000001",
      "",
      ".ds-logs-endpoint.events.process-default-2021.12.09-000001",
      ""
    ],
    "agent.type": [
      "endpoint"
    ],
    "signal.rule.language": [
      "eql"
    ],
    "user.domain": [
      "DESKTOP-QBBSCUT"
    ],
    "host.id": [
      "4143c277-074e-47a9-b37d-37f94b508705"
    ],
    "kibana.alert.rule.max_signals": [
      100
    ],
    "kibana.alert.rule.risk_score": [
      21
    ],
    "process.code_signature.status": [
      "trusted"
    ],
    "signal.original_event.dataset": [
      "endpoint.events.process"
    ],
    "kibana.alert.rule.consumer": [
      "siem"
    ],
    "kibana.alert.rule.category": [
      "Event Correlation Rule"
    ],
    "host.os.Ext.variant": [
      "Windows 10 Pro"
    ],
    "@timestamp": [
      "2021-12-09T12:32:59.110Z"
    ],
    "signal.rule.updated_by": [
      "elastic"
    ],
    "host.os.platform": [
      "windows"
    ],
    "kibana.alert.rule.severity": [
      "low"
    ],
    "kibana.alert.original_event.agent_id_status": [
      "verified"
    ],
    "data_stream.dataset": [
      "endpoint.events.process"
    ],
    "kibana.alert.rule.meta.kibana_siem_app_url": [
      "https://snapshot-7d0fa6.kb.us-central1.gcp.qa.cld.elstc.co:9243/app/security"
    ],
    "kibana.version": [
      "8.0.0-SNAPSHOT"
    ],
    "signal.rule.license": [
      ""
    ],
    "signal.ancestors.type": [
      "event",
      "signal",
      "event",
      "signal"
    ],
    "kibana.alert.ancestors.rule": [
      "21535bc0-58ec-11ec-a41a-131fd10250bd",
      "21535bc0-58ec-11ec-a41a-131fd10250bd"
    ],
    "kibana.alert.rule.rule_id": [
      "89937845-488e-4eda-b9b4-53699356f5e5"
    ],
    "signal.rule.query": [
      "sequence [ process where process.name == \"cmd.exe\" ] [ process where process.name == \"notepad.exe\" ]"
    ],
    "signal.rule.type": [
      "eql"
    ],
    "kibana.alert.ancestors.id": [
      "lNzVnX0BWpFDgqW3ZMS_",
      "eae12e458361a5da7e57509252a0c1e8fce99e364fbd5e85425a57666df4e349",
      "RV5Enn0BCj1zhlBeakic",
      "cd8dcbec32d44a5737d2bd077e28d7df1a6276b61126cb287be7efeecd7a19f0"
    ],
    "host.os.full": [
      "Windows 10 Pro 1903 (10.0.18362.1256)"
    ],
    "kibana.alert.rule.description": [
      "test"
    ],
    "kibana.alert.rule.producer": [
      "siem"
    ],
    "signal.rule.created_by": [
      "elastic"
    ],
    "kibana.alert.rule.to": [
      "now"
    ],
    "signal.rule.interval": [
      "5s"
    ],
    "kibana.alert.rule.created_by": [
      "elastic"
    ],
    "signal.rule.id": [
      "21535bc0-58ec-11ec-a41a-131fd10250bd"
    ],
    "process.code_signature.subject_name": [
      "Microsoft Windows"
    ],
    "signal.reason": [
      "event by zeus on DESKTOP-QBBSCUT created low alert EQL rule."
    ],
    "host.os.name": [
      "Windows"
    ],
    "kibana.alert.rule.name": [
      "EQL rule"
    ],
    "kibana.alert.rule.language": [
      "eql"
    ],
    "host.name": [
      "DESKTOP-QBBSCUT"
    ],
    "signal.status": [
      "open"
    ],
    "event.kind": [
      "signal",
      "signal"
    ],
    "process.code_signature.trusted": [
      true
    ],
    "signal.rule.tags": [
      "qa"
    ],
    "signal.rule.created_at": [
      "2021-12-09T12:32:54.537Z"
    ],
    "kibana.alert.workflow_status": [
      "open"
    ],
    "kibana.alert.rule.uuid": [
      "21535bc0-58ec-11ec-a41a-131fd10250bd"
    ],
    "kibana.alert.reason": [
      "event by zeus on DESKTOP-QBBSCUT created low alert EQL rule."
    ],
    "data_stream.type": [
      "logs"
    ],
    "signal.ancestors.id": [
      "lNzVnX0BWpFDgqW3ZMS_",
      "eae12e458361a5da7e57509252a0c1e8fce99e364fbd5e85425a57666df4e349",
      "RV5Enn0BCj1zhlBeakic",
      "cd8dcbec32d44a5737d2bd077e28d7df1a6276b61126cb287be7efeecd7a19f0"
    ],
    "signal.original_time": [
      "2021-12-09T06:15:36.660Z"
    ],
    "ecs.version": [
      "1.11.0"
    ],
    "kibana.alert.ancestors.index": [
      ".ds-logs-endpoint.events.process-default-2021.12.09-000001",
      "",
      ".ds-logs-endpoint.events.process-default-2021.12.09-000001",
      ""
    ],
    "kibana.alert.depth": [
      2
    ],
    "agent.version": [
      "8.0.0-SNAPSHOT"
    ],
    "host.os.family": [
      "windows"
    ],
    "kibana.alert.rule.from": [
      "now-108000005s"
    ],
    "kibana.alert.rule.query": [
      "sequence [ process where process.name == \"cmd.exe\" ] [ process where process.name == \"notepad.exe\" ]"
    ],
    "signal.rule.version": [
      "1"
    ],
    "signal.original_event.kind": [
      "event"
    ],
    "kibana.alert.status": [
      "active"
    ],
    "signal.rule.index": [
      "apm-*-transaction*",
      "traces-apm*",
      "auditbeat-*",
      "endgame-*",
      "filebeat-*",
      "logs-*",
      "packetbeat-*",
      "winlogbeat-*"
    ],
    "signal.depth": [
      2
    ],
    "kibana.alert.original_event.dataset": [
      "endpoint.events.process"
    ],
    "signal.rule.immutable": [
      "false"
    ],
    "kibana.alert.rule.rule_type_id": [
      "siem.eqlRule"
    ],
    "signal.rule.name": [
      "EQL rule"
    ],
    "event.module": [
      "endpoint"
    ],
    "signal.rule.rule_id": [
      "89937845-488e-4eda-b9b4-53699356f5e5"
    ],
    "host.os.kernel": [
      "1903 (10.0.18362.1256)"
    ],
    "kibana.alert.rule.index": [
      "apm-*-transaction*",
      "traces-apm*",
      "auditbeat-*",
      "endgame-*",
      "filebeat-*",
      "logs-*",
      "packetbeat-*",
      "winlogbeat-*"
    ],
    "kibana.alert.rule.license": [
      ""
    ],
    "kibana.alert.original_event.kind": [
      "event"
    ],
    "kibana.alert.rule.updated_at": [
      "2021-12-09T12:32:56.517Z"
    ],
    "signal.rule.description": [
      "test"
    ],
    "data_stream.namespace": [
      "default"
    ],
    "message": [
      "Endpoint process event"
    ],
    "signal.rule.to": [
      "now"
    ],
    "kibana.alert.rule.created_at": [
      "2021-12-09T12:32:54.537Z"
    ],
    "kibana.space_ids": [
      "default"
    ],
    "kibana.alert.rule.meta.from": [
      "30000h"
    ],
    "event.dataset": [
      "endpoint.events.process"
    ],
    "kibana.alert.original_time": [
      "2021-12-09T06:15:36.660Z"
    ]
  }
}
@ghost ghost added bug Fixes for quality problems that affect the customer experience triage_needed Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Dec 9, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@ghost ghost added the impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. label Dec 9, 2021
@manishgupta-qasource
Copy link

Reviewed & assigned to @MadameSheema

@MadameSheema
Copy link
Member

@karanbirsingh-qasource can you please let us know which data views were selected in the timeline? Thanks

@ghost
Copy link
Author

ghost commented Dec 9, 2021

@karanbirsingh-qasource can you please let us know which data views were selected in the timeline? Thanks

Security Default data view
image

@MadameSheema
Copy link
Member

@karanbirsingh-qasource can you please take the same screenshot after clicking on Advanced options in order to see the data views? thank you

@ghost
Copy link
Author

ghost commented Dec 13, 2021

@karanbirsingh-qasource can you please take the same screenshot after clicking on Advanced options in order to see the data views? thank you

Sure @MadameSheema please find below the required screen-cast

image

sequence.mp4

@MadameSheema
Copy link
Member

@karanbirsingh-qasource can you please validate the fix on the latest 8.0.0? Thanks!

@ghost
Copy link
Author

ghost commented Jan 14, 2022

Hi @MadameSheema

We have validated this issue on 8.0.0-Branch and found that issue is still occuring 🔴 .

Build Details:

Version: 8.0.0.-SNAPSHOT
Build:9007199254740991

Screen-Cast:

image

correlation.mp4

thanks !!

@MadameSheema
Copy link
Member

@karanbirsingh-qasource the fix was validated with the snapshot or with the latest 8.0 branch?

@ghost
Copy link
Author

ghost commented Jan 14, 2022

Latest 8.0 branch @MadameSheema

@ghost
Copy link
Author

ghost commented Jan 14, 2022

Latest 8.0.0-SNAPSHOT Staging Cloud has also issue occuring observation.

Also JFI 8.0.0-SNAPSHOT build creation are failing on Release ( LA region ) and QA Cloud Platform

Build Details:

Commit:2fa075fc23e8e5e78c862cd6518fdcd3430ae1f7
Build:48933

image

@MadameSheema
Copy link
Member

@karanbirsingh-qasource can you please check if the issue you are facing now could be related to the following one? #122958 Thanks!

@ghost
Copy link
Author

ghost commented Jan 17, 2022

@MadameSheema i have checked this issue and its been present on old builds too as in 7.16.3 so i am not sure this #122958 can be the reason for this issue . However we will be keeping any eye on this issue too.

image

@MadameSheema
Copy link
Member

@karanbirsingh-qasource can you please check it again on 8.0 and also in main branch? Thanks

@ghost
Copy link
Author

ghost commented Jan 17, 2022

HI @MadameSheema issue is still occuring on 8.0.0-SNAPSHOT and main branch too.

  • main branch

image

event.mp4
  • 8.0.0-SNAPSHOT
Commit:155e06787e48de9a8de4345d86a826e95edf32ec
Build:49040

image

@MadameSheema
Copy link
Member

The current behavior of the above issue has changed. Now we display an alert but we don't display the sequence as a block since we are not filtering by the expected group id value. This is directly related with #123370

@MadameSheema
Copy link
Member

@deepikakeshav-qasource @manishgupta-qasource can you please help to coordinate the testing of this on 8.0 latest branch? This is top priority, thanks :)

@ghost
Copy link

ghost commented Jan 25, 2022

Hi @MadameSheema

We have validated this issue on 8.0.0 branch and observed that issue is still Occurring.

Please find below testing details:

Build Details:

Version: 8.0.0 branch
commit:61850b11e7ea585630fec1ed981a7715e8b7c1bb

Screencast:

eql_rule.mp4

Thanks!!

@ghost ghost reopened this Jan 25, 2022
@kqualters-elastic
Copy link
Contributor

kqualters-elastic commented Jan 25, 2022

@deepikakeshav-qasource if you create a new timeline to clear out any existing results and then open an eql sequence, does the data appear?

@MadameSheema
Copy link
Member

@kqualters-elastic I think it could be related to a different issue we are facing in new environments where the .alerts index is not selected by default since I was able to test this on latest 8.0 branch and was working fine for me.

@ghost
Copy link
Author

ghost commented Jan 27, 2022

Hi @MadameSheema & @kqualters-elastic

we have validated this issue on 8.0 Branch as well as 8.0.0-RC2-BC2 and found it fixed 🟢 .

please find below details:

Build Details:

version: 8.0.0-RC2-BC2
commit:01d4a6a088cae588e542623a47054bc76cfb2a6d
build:49157

Branch:

image

Screen-Cast

actual-Fixed.mp4

Hence we are closing this issue.

thanks !!

@ghost ghost closed this as completed Jan 27, 2022
@ghost ghost added the QA:Validated Issue has been validated by QA label Jan 27, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. QA:Validated Issue has been validated by QA Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting:Investigations Security Solution Investigations Team v8.0.0
Projects
None yet
5 participants