-
Notifications
You must be signed in to change notification settings - Fork 493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FR] Add limited support and validation for version and revision #3657
[FR] Add limited support and validation for version and revision #3657
Conversation
…s-in-config-files' into 3619-frdac-further-decouple-reliance-on-default-rule-dir-locations
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
Testing again with recent changes: Bypass Version Lock False 🟢
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev build-release --update-version-lock
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/forteea1/Code/clean_mains/detection-rules/detection_rules/__main__.py", line 34, in <module>
main()
File "/home/forteea1/Code/clean_mains/detection-rules/detection_rules/__main__.py", line 31, in main
root(prog_name="detection_rules")
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/detection_rules/devtools.py", line 98, in build_release
registry_data = config['registry_data']
~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'registry_data'
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev deprecate-rule custom_rules/rules/command_and_control_test_potential_protocol_tunneling_via_chisel_client.toml
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Rule has not been version locked and so does not need to be deprecated. Delete the file or update the maturity to `development` instead.
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev update-lock-versions
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Are you sure you want to update hashes for 4 rules without a version bump? [y/N]: Y
Rule changes detected!
- 0 changed rules
- 4 new rules
- 0 newly deprecated rules
Detailed changes:
Updated /home/forteea1/Code/clean_mains/detection-rules/custom_rules/etc/version.lock.json file
No hashes updated
❯ python -m detection_rules dev trim-version-lock 8.15.0
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Loading rules ...
No changes
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev trim-version-lock 8.9.0
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Loading rules ...
No changes
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules kibana export-rules -d mikas_folder -s
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
- skipping DAC Demo Dev Rule 1 - also updated - ValidationError
- skipping DAC Demo Dev Rule 13 - ValidationError
- skipping DAC Demo Dev Rule 1 - ValidationError
- skipping TEST Potential Protocol Tunneling via Chisel Client - ValidationError
4 rules exported
0 rules converted
0 saved to mikas_folder
4 errors saved to mikas_folder/_errors.txt
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 2s
❯ python -m detection_rules kibana export-rules -d custom_rules -s
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
- skipping DAC Demo Dev Rule 1 - also updated - ValidationError
- skipping DAC Demo Dev Rule 13 - ValidationError
- skipping DAC Demo Dev Rule 1 - ValidationError
- skipping TEST Potential Protocol Tunneling via Chisel Client - ValidationError
4 rules exported
0 rules converted
0 saved to custom_rules
4 errors saved to custom_rules/_errors.txt
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 2s
❯ cat custom_rules/_errors.txt
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: custom_rules/_errors.txt
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ - DAC Demo Dev Rule 1 - also updated - {'rule': [ValidationError({'_schema': ['Configuration error: Rule DAC Demo Dev
│ Rule 1 - also updated - 09ea590e-69d1-44de-a386-3d84d85cc2f2 should not contain rules with `version and revision` set.
│ ']}), ValidationError({'type': ['Must be equal to esql.'], 'language': ['Must be equal to esql.']}), ValidationError({
│ 'type': ['Must be equal to threshold.'], 'threshold': ['Missing data for required field.']}), ValidationError({'type':
│ ['Must be equal to threat_match.'], 'threat_mapping': ['Missing data for required field.'], 'threat_index': ['Missing
│ data for required field.']}), ValidationError({'type': ['Must be equal to machine_learning.'], 'anomaly_threshold': [
│ 'Missing data for required field.'], 'machine_learning_job_id': ['Missing data for required field.']}), ValidationErro
│ r({'type': ['Must be equal to query.']}), ValidationError({'type': ['Must be equal to new_terms.'], 'new_terms': ['Mis
│ sing data for required field.']})]}
2 │ - DAC Demo Dev Rule 13 - {'rule': [ValidationError({'_schema': ['Configuration error: Rule DAC Demo Dev Rule 13 - 222e
│ 1b03-fdc9-42a5-911e-2e3e0533549a should not contain rules with `version and revision` set.']}), ValidationError({'type
│ ': ['Must be equal to esql.'], 'language': ['Must be equal to esql.']}), ValidationError({'type': ['Must be equal to t
│ hreshold.'], 'threshold': ['Missing data for required field.']}), ValidationError({'type': ['Must be equal to threat_m
│ atch.'], 'threat_mapping': ['Missing data for required field.'], 'threat_index': ['Missing data for required field.']}
│ ), ValidationError({'type': ['Must be equal to machine_learning.'], 'anomaly_threshold': ['Missing data for required f
│ ield.'], 'machine_learning_job_id': ['Missing data for required field.']}), ValidationError({'type': ['Must be equal t
│ o query.']}), ValidationError({'type': ['Must be equal to new_terms.'], 'new_terms': ['Missing data for required field
│ .']})]}
3 │ - DAC Demo Dev Rule 1 - {'rule': [ValidationError({'_schema': ['Configuration error: Rule DAC Demo Dev Rule 1 - 794d2f
│ c0-ecd0-4963-99da-fd587666b80d should not contain rules with `version and revision` set.']}), ValidationError({'type':
│ ['Must be equal to esql.'], 'language': ['Must be equal to esql.']}), ValidationError({'type': ['Must be equal to thr
│ eshold.'], 'threshold': ['Missing data for required field.']}), ValidationError({'type': ['Must be equal to threat_mat
│ ch.'], 'threat_mapping': ['Missing data for required field.'], 'threat_index': ['Missing data for required field.']}),
│ ValidationError({'type': ['Must be equal to machine_learning.'], 'anomaly_threshold': ['Missing data for required fie
│ ld.'], 'machine_learning_job_id': ['Missing data for required field.']}), ValidationError({'type': ['Must be equal to
│ query.']}), ValidationError({'type': ['Must be equal to new_terms.'], 'new_terms': ['Missing data for required field.'
│ ]})]}
4 │ - TEST Potential Protocol Tunneling via Chisel Client - {'rule': [ValidationError({'_schema': ['Configuration error: R
│ ule TEST Potential Protocol Tunneling via Chisel Client - e8e3af2a-11b8-4ab7-9ca1-c6db621ea89d should not contain rule
│ s with `version and revision` set.']}), ValidationError({'type': ['Must be equal to esql.'], 'language': ['Must be equ
│ al to esql.']}), ValidationError({'type': ['Must be equal to threshold.'], 'threshold': ['Missing data for required fi
│ eld.']}), ValidationError({'type': ['Must be equal to threat_match.'], 'threat_mapping': ['Missing data for required f
│ ield.'], 'threat_index': ['Missing data for required field.']}), ValidationError({'type': ['Must be equal to machine_l
│ earning.'], 'anomaly_threshold': ['Missing data for required field.'], 'machine_learning_job_id': ['Missing data for r
│ equired field.']}), ValidationError({'type': ['Must be equal to query.']}), ValidationError({'type': ['Must be equal t
│ o new_terms.'], 'new_terms': ['Missing data for required field.']})]}
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules kibana export-rules -d custom_rules/rules -s -sv
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
4 rules exported
4 rules converted
4 saved to custom_rules/rules
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 3s
❯ python -m detection_rules kibana import-rules -o -d custom_rules/rules/
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
4 rule(s) successfully imported
- 794d2fc0-ecd0-4963-99da-fd587666b80d
- 222e1b03-fdc9-42a5-911e-2e3e0533549a
- e8e3af2a-11b8-4ab7-9ca1-c6db621ea89d
- 09ea590e-69d1-44de-a386-3d84d85cc2f2
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 5s
❯ python -m detection_rules import-rules-to-repo /home/forteea1/Code/dac_demo/detection-rules-dac-demo/exports/20240518T115752L.ndjson -s custom_rules/rules --required-only
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[+] Building rule for custom_rules/rules/dac_demo_dev_rule_1_updated.toml
[+] Building rule for custom_rules/rules/dac_demo_dev_rule_2_feedback.toml
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules export-rules-from-repo
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Exported 6 rules into /home/forteea1/Code/clean_mains/detection-rules/exports/20240523T161054L.ndjson
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules view-rule rules/cross-platform/threat_intel_indicator_match_address.toml
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
{
"author": [
"Elastic"
],
"description": "This rule is triggered when an IP address indicator from the Threat Intel Filebeat module or integrations has a match against a network event.",
"from": "now-65m",
"index": [
"auditbeat-*",
"endgame-*",
"filebeat-*",
"logs-*",
"packetbeat-*",
"winlogbeat-*"
],
"interval": "1h",
"language": "kuery",
"license": "Elastic License v2",
"name": "Threat Intel IP Address Indicator Match",
"note": "## Triage and Analysis\n\n### Investigating Threat Intel IP Address Indicator Match\n\nThreat Intel indicator match rules allow matching from a local observation, such as an endpoint event that records a file hash with an entry of a file hash stored within the Threat Intel integrations index.\n\nMatches are based on threat intelligence data that's been ingested during the last 30 days. Some integrations don't place expiration dates on their threat indicators, so we strongly recommend validating ingested threat indicators and reviewing match results. When reviewing match results, check associated activity to determine whether the event requires additional investigation.\n\nThis rule is triggered when an IP address indicator from the Threat Intel Filebeat module or a threat intelligence integration matches against a network event.\n\n> **Note**:\n> This investigation guide uses the [Osquery Markdown Plugin](https://www.elastic.co/guide/en/security/master/invest-guide-run-osquery.html) introduced in Elastic Stack version 8.5.0. Older Elastic Stack versions will display unrendered Markdown in this guide.\n\n#### Possible investigation steps\n\n- Gain context about the field that matched the local observation so you can understand the nature of the connection. This information can be found in the `threat.indicator.matched.field` field.\n- Investigate the IP address, which can be found in the `threat.indicator.matched.atomic` field:\n - Check the reputation of the IP address in resources like VirusTotal, Hybrid-Analysis, CISCO Talos, Any.run, etc.\n - Execute a reverse DNS lookup to retrieve hostnames associated with the given IP address.\n- Assess whether this behavior is prevalent in the environment by looking for similar occurrences across hosts.\n- Identify the process responsible for the connection, and investigate the process execution chain (parent process tree) for unknown processes. Examine their executable files for prevalence, whether they are located in expected locations, and if they are signed with valid digital signatures.\n- Retrieve the involved process executable and examine the host for derived artifacts that indicate suspicious activities:\n - Analyze the process executable using a private sandboxed analysis system.\n - Observe and collect information about the following activities in both the sandbox and the alert subject host:\n - Attempts to contact external domains and addresses.\n - Use the Elastic Defend network events to determine domains and addresses contacted by the subject process by filtering by the process' `process.entity_id`.\n - Examine the DNS cache for suspicious or anomalous entries.\n - !{osquery{\"label\":\"Osquery - Retrieve DNS Cache\",\"query\":\"SELECT * FROM dns_cache\"}}\n - Use the Elastic Defend registry events to examine registry keys accessed, modified, or created by the related processes in the process tree.\n - Examine the host services for suspicious or anomalous entries.\n - !{osquery{\"label\":\"Osquery - Retrieve All Services\",\"query\":\"SELECT description, display_name, name, path, pid, service_type, start_type, status, user_account FROM services\"}}\n - !{osquery{\"label\":\"Osquery - Retrieve Services Running on User Accounts\",\"query\":\"SELECT description, display_name, name, path, pid, service_type, start_type, status, user_account FROM services WHERE\\nNOT (user_account LIKE '%LocalSystem' OR user_account LIKE '%LocalService' OR user_account LIKE '%NetworkService' OR\\nuser_account == null)\\n\"}}\n - !{osquery{\"label\":\"Osquery - Retrieve Service Unsigned Executables with Virustotal Link\",\"query\":\"SELECT concat('https://www.virustotal.com/gui/file/', sha1) AS VtLink, name, description, start_type, status, pid,\\nservices.path FROM services JOIN authenticode ON services.path = authenticode.path OR services.module_path =\\nauthenticode.path JOIN hash ON services.path = hash.path WHERE authenticode.result != 'trusted'\\n\"}}\n- Using the data collected through the analysis, scope users targeted and other machines infected in the environment.\n\n### False Positive Analysis\n\n- When a match is found, it's important to consider the indicator's initial release date. Threat intelligence is useful for augmenting existing security processes but can quickly become outdated. In other words, some threat intelligence only represents a specific set of activity observed at a specific time. For example, an IP address may have hosted malware observed in a Dridex campaign months ago, but it's possible that IP has been remediated and no longer represents any threat.\n- False positives might occur after large and publicly written campaigns if curious employees interact with attacker infrastructure.\n- Some feeds may include internal or known benign addresses by mistake (e.g., 8.8.8.8, google.com, 127.0.0.1, etc.). Make sure you understand how blocking a specific domain or address might impact the organization or normal system functioning.\n\n### Response and Remediation\n\n- Initiate the incident response process based on the outcome of the triage.\n- Isolate the involved host to prevent further post-compromise behavior.\n- If the triage identified malware, search the environment for additional compromised hosts.\n - Implement temporary network rules, procedures, and segmentation to contain the malware.\n - Stop suspicious processes.\n - Immediately block the identified indicators of compromise (IoCs).\n - Inspect the affected systems for additional malware backdoors like reverse shells, reverse proxies, or droppers that attackers could use to reinfect the system.\n- Remove and block malicious artifacts identified during triage.\n- Run a full antimalware scan. This may reveal additional artifacts left in the system, persistence mechanisms, and malware components.\n- Determine the initial vector abused by the attacker and take action to prevent reinfection through the same vector.\n- Using the incident response data, update logging and audit policies to improve the mean time to detect (MTTD) and the mean time to respond (MTTR).\n",
"query": "source.ip:* or destination.ip:*\n",
"references": [
"https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-threatintel.html",
"https://www.elastic.co/guide/en/security/master/es-threat-intel-integrations.html",
"https://www.elastic.co/security/tip"
],
"required_fields": [
{
"ecs": true,
"name": "destination.ip",
"type": "ip"
},
{
"ecs": true,
"name": "source.ip",
"type": "ip"
}
],
"risk_score": 99,
"rule_id": "0c41e478-5263-4c69-8f9e-7dfd2c22da64",
"setup": "## Setup\n\nThis rule needs threat intelligence indicators to work.\nThreat intelligence indicators can be collected using an [Elastic Agent integration](https://www.elastic.co/guide/en/security/current/es-threat-intel-integrations.html#agent-ti-integration),\nthe [Threat Intel module](https://www.elastic.co/guide/en/security/current/es-threat-intel-integrations.html#ti-mod-integration),\nor a [custom integration](https://www.elastic.co/guide/en/security/current/es-threat-intel-integrations.html#custom-ti-integration).\n\nMore information can be found [here](https://www.elastic.co/guide/en/security/current/es-threat-intel-integrations.html).\n",
"severity": "critical",
"tags": [
"OS: Windows",
"Data Source: Elastic Endgame",
"Rule Type: Indicator Match"
],
"threat_filters": [
{
"$state": {
"store": "appState"
},
"meta": {
"disabled": false,
"key": "event.category",
"negate": false,
"params": {
"query": "threat"
},
"type": "phrase"
},
"query": {
"match_phrase": {
"event.category": "threat"
}
}
},
{
"$state": {
"store": "appState"
},
"meta": {
"disabled": false,
"key": "event.kind",
"negate": false,
"params": {
"query": "enrichment"
},
"type": "phrase"
},
"query": {
"match_phrase": {
"event.kind": "enrichment"
}
}
},
{
"$state": {
"store": "appState"
},
"meta": {
"disabled": false,
"key": "event.type",
"negate": false,
"params": {
"query": "indicator"
},
"type": "phrase"
},
"query": {
"match_phrase": {
"event.type": "indicator"
}
}
}
],
"threat_index": [
"filebeat-*",
"logs-ti_*"
],
"threat_indicator_path": "threat.indicator",
"threat_language": "kuery",
"threat_mapping": [
{
"entries": [
{
"field": "source.ip",
"type": "mapping",
"value": "threat.indicator.ip"
}
]
},
{
"entries": [
{
"field": "destination.ip",
"type": "mapping",
"value": "threat.indicator.ip"
}
]
}
],
"threat_query": "@timestamp >= \"now-30d/d\" and event.module:(threatintel or ti_*) and threat.indicator.ip:* and not labels.is_ioc_transform_source:\"true\"",
"timeline_id": "495ad7a7-316e-4544-8a0f-9c098daee76e",
"timeline_title": "Generic Threat Match Timeline",
"timestamp_override": "event.ingested",
"type": "threat_match",
"version": 1
}
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules view-rule custom_rules/rules/command_and_control_dac_demo_dev_rule_1.toml --api-format
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
{
"actions": [],
"author": [
"DAC User"
],
"description": "Test Rule",
"enabled": true,
"exceptions_list": [],
"false_positives": [],
"from": "now-9m",
"index": [
"logs-endpoint.events.*"
],
"interval": "5m",
"language": "eql",
"license": "Elastic License v2",
"max_signals": 100,
"name": "DAC Demo Dev Rule 1",
"note": "Test Note",
"query": "process where host.os.type == \"linux\" and process.name == \"updated\"\n",
"references": [
"https://dac-reference.readthedocs.io/en/latest/"
],
"related_integrations": [
{
"package": "endpoint",
"version": "^8.2.0"
}
],
"required_fields": [
{
"ecs": true,
"name": "host.os.type",
"type": "keyword"
},
{
"ecs": true,
"name": "process.name",
"type": "keyword"
}
],
"risk_score": 47,
"risk_score_mapping": [],
"rule_id": "794d2fc0-ecd0-4963-99da-fd587666b80d",
"setup": "Test Setup",
"severity": "medium",
"severity_mapping": [],
"tags": [
"Domain: Endpoint",
"OS: Linux",
"Use Case: Threat Detection",
"Tactic: Command and Control",
"Data Source: Elastic Defend"
],
"threat": [
{
"framework": "MITRE ATT&CK",
"tactic": {
"id": "TA0011",
"name": "Command and Control",
"reference": "https://attack.mitre.org/tactics/TA0011/"
},
"technique": [
{
"id": "T1572",
"name": "Protocol Tunneling",
"reference": "https://attack.mitre.org/techniques/T1572/"
}
]
}
],
"timestamp_override": "event.ingested",
"to": "now",
"type": "eql",
"version": 1
}
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules view-rule custom_rules/rules/command_and_control_dac_demo_dev_rule_1.toml --rule-format
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[metadata]
creation_date = "2024/05/23"
maturity = "production"
updated_date = "2024/05/23"
[rule]
actions = []
author = ["DAC User"]
description = "Test Rule"
enabled = true
exceptions_list = []
false_positives = []
from = "now-9m"
index = ["logs-endpoint.events.*"]
interval = "5m"
language = "eql"
license = "Elastic License v2"
max_signals = 100
name = "DAC Demo Dev Rule 1"
note = "Test Note"
references = ["https://dac-reference.readthedocs.io/en/latest/"]
risk_score = 47
risk_score_mapping = []
rule_id = "794d2fc0-ecd0-4963-99da-fd587666b80d"
setup = "Test Setup"
severity = "medium"
severity_mapping = []
tags = [
"Domain: Endpoint",
"OS: Linux",
"Use Case: Threat Detection",
"Tactic: Command and Control",
"Data Source: Elastic Defend",
]
timestamp_override = "event.ingested"
to = "now"
type = "eql"
query = '''
process where host.os.type == "linux" and process.name == "updated"
'''
[[rule.related_integrations]]
package = "endpoint"
version = "^8.2.0"
[[rule.required_fields]]
ecs = true
name = "host.os.type"
type = "keyword"
[[rule.required_fields]]
ecs = true
name = "process.name"
type = "keyword"
[[rule.threat]]
framework = "MITRE ATT&CK"
[[rule.threat.technique]]
id = "T1572"
name = "Protocol Tunneling"
reference = "https://attack.mitre.org/techniques/T1572/"
[rule.threat.tactic]
id = "TA0011"
name = "Command and Control"
reference = "https://attack.mitre.org/tactics/TA0011/"
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules view-rule custom_rules/rules/command_and_control_dac_demo_dev_rule_1.toml --rule-format^C
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev build-release --update-version-lock
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/forteea1/Code/clean_mains/detection-rules/detection_rules/__main__.py", line 34, in <module>
main()
File "/home/forteea1/Code/clean_mains/detection-rules/detection_rules/__main__.py", line 31, in main
root(prog_name="detection_rules")
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/env/detection-rules-build/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/forteea1/Code/clean_mains/detection-rules/detection_rules/devtools.py", line 101, in build_release
assert 'registry_data' in config, err_msg
^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: No `registry_data` in package config. Please see the /home/forteea1/Code/clean_mains/detection-rules/detection_rules/etc/package.yaml file for an example on how to supply this field in /home/forteea1/Code/clean_mains/detection-rules/custom_rules/etc/packages.yaml.
Version Lock True 🟢
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev build-release --update-version-lock
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
WARNING: You cannot run this command when the versioning strategy is configured to bypass the version lock. Set `bypass_version_lock` to `False` in the rules config to use the version lock.
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev deprecate-rule custom_rules/rules/command_and_control_test_potential_protocol_tunneling_via_chisel_client.toml
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Rule moved to /home/forteea1/Code/clean_mains/detection-rules/custom_rules/rules/_deprecated/command_and_control_test_potential_protocol_tunneling_via_chisel_client.toml - remember to git add this file
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules dev update-lock-versions
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Are you sure you want to update hashes for 3 rules without a version bump? [y/N]: Y
WARNING: You cannot run this command when the versioning strategy is configured to bypass the version lock. Set `bypass_version_lock` to `False` in the rules config to use the version lock.
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 8s
❯ python -m detection_rules dev trim-version-lock 8.9.0
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
WARNING: Cannot trim the version lock when the versioning strategy is configured to bypass the version lock. Set `bypass_version_lock` to `false` in the rules config to use the version lock.
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules kibana export-rules -d custom_rules/rules -s -sv
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
4 rules exported
4 rules converted
4 saved to custom_rules/rules
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 4s
❯ python -m detection_rules kibana export-rules -d custom_rules/rules -s
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
4 rules exported
4 rules converted
4 saved to custom_rules/rules
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 2s
❯ python -m detection_rules kibana import-rules -o -d custom_rules/rules/
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
6 rule(s) successfully imported
- 794d2fc0-ecd0-4963-99da-fd587666b80d
- 222e1b03-fdc9-42a5-911e-2e3e0533549a
- 230e7043-b0fc-44eb-86b8-e9eb1e5e38d4
- 09ea590e-69d1-44de-a386-3d84d85cc2f2
- 69b96ef7-f00f-473b-9525-81123329a036
- e8e3af2a-11b8-4ab7-9ca1-c6db621ea89d
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 8s
❯ python -m detection_rules import-rules-to-repo /home/forteea1/Code/dac_demo/detection-rules-dac-demo/exports/20240518T115752L.ndjson -s custom_rules/rules --required-only
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[+] Building rule for custom_rules/rules/dac_demo_dev_rule_1_updated.toml
[+] Building rule for custom_rules/rules/dac_demo_dev_rule_2_feedback.toml
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules export-rules-from-repo
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Exported 6 rules into /home/forteea1/Code/clean_mains/detection-rules/exports/20240523T161813L.ndjson
…pport-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte took 2s
❯ python -m detection_rules view-rule custom_rules/rules/command_and_control_dac_demo_dev_rule_1.toml --rule-format
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[metadata]
creation_date = "2024/05/23"
maturity = "production"
updated_date = "2024/05/23"
[rule]
actions = []
author = ["DAC User"]
description = "Test Rule"
enabled = true
exceptions_list = []
false_positives = []
from = "now-9m"
index = ["logs-endpoint.events.*"]
interval = "5m"
language = "eql"
license = "Elastic License v2"
max_signals = 100
name = "DAC Demo Dev Rule 1"
note = "Test Note"
references = ["https://dac-reference.readthedocs.io/en/latest/"]
revision = 0
risk_score = 47
risk_score_mapping = []
rule_id = "52e84641-7483-4a00-a301-13baffa51b5f"
setup = "Test Setup"
severity = "medium"
severity_mapping = []
tags = [
"Domain: Endpoint",
"OS: Linux",
"Use Case: Threat Detection",
"Tactic: Command and Control",
"Data Source: Elastic Defend",
]
timestamp_override = "event.ingested"
to = "now"
type = "eql"
version = 1
query = '''
process where host.os.type == "linux" and process.name == "updated"
'''
[[rule.related_integrations]]
package = "endpoint"
version = "^8.2.0"
[[rule.required_fields]]
ecs = true
name = "host.os.type"
type = "keyword"
[[rule.required_fields]]
ecs = true
name = "process.name"
type = "keyword"
[[rule.threat]]
framework = "MITRE ATT&CK"
[[rule.threat.technique]]
id = "T1572"
name = "Protocol Tunneling"
reference = "https://attack.mitre.org/techniques/T1572/"
[rule.threat.tactic]
id = "TA0011"
name = "Command and Control"
reference = "https://attack.mitre.org/tactics/TA0011/"
…mited-support-for-version-and-revision-to-baseruledata [!?] is v0.1.0 via v3.12.3 (detection-rules-build) on eric.forte
❯ python -m detection_rules view-rule custom_rules/rules/command_and_control_dac_demo_dev_rule_1.toml --api-format
Loaded config file: /home/forteea1/Code/clean_mains/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
{
"actions": [],
"author": [
"DAC User"
],
"description": "Test Rule",
"enabled": true,
"exceptions_list": [],
"false_positives": [],
"from": "now-9m",
"index": [
"logs-endpoint.events.*"
],
"interval": "5m",
"language": "eql",
"license": "Elastic License v2",
"max_signals": 100,
"name": "DAC Demo Dev Rule 1",
"note": "Test Note",
"query": "process where host.os.type == \"linux\" and process.name == \"updated\"\n",
"references": [
"https://dac-reference.readthedocs.io/en/latest/"
],
"related_integrations": [
{
"package": "endpoint",
"version": "^8.2.0"
}
],
"required_fields": [
{
"ecs": true,
"name": "host.os.type",
"type": "keyword"
},
{
"ecs": true,
"name": "process.name",
"type": "keyword"
}
],
"revision": 0,
"risk_score": 47,
"risk_score_mapping": [],
"rule_id": "52e84641-7483-4a00-a301-13baffa51b5f",
"setup": "Test Setup",
"severity": "medium",
"severity_mapping": [],
"tags": [
"Domain: Endpoint",
"OS: Linux",
"Use Case: Threat Detection",
"Tactic: Command and Control",
"Data Source: Elastic Defend"
],
"threat": [
{
"framework": "MITRE ATT&CK",
"tactic": {
"id": "TA0011",
"name": "Command and Control",
"reference": "https://attack.mitre.org/tactics/TA0011/"
},
"technique": [
{
"id": "T1572",
"name": "Protocol Tunneling",
"reference": "https://attack.mitre.org/techniques/T1572/"
}
]
}
],
"timestamp_override": "event.ingested",
"to": "now",
"type": "eql",
"version": 1
}
|
…rsion-and-revision-to-baseruledata
detection_rules/version_lock.py
Outdated
@@ -64,6 +64,16 @@ def __getitem__(self, item) -> VersionLockFileEntry: | |||
raise KeyError(item) | |||
return self.data[item] | |||
|
|||
@classmethod | |||
def from_dict(cls: Type[ClassT], obj: dict) -> ClassT: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we still need these methods?
"""Assemble all the rules into Kibana-ready release files.""" | ||
config = load_dump(str(config_file))['package'] | ||
if RULES_CONFIG.bypass_version_lock: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note for future: we can cleanup all these calls with a bypass_check
decorator for applicable functions
detection_rules/devtools.py
Outdated
if not deprecation_folder: | ||
deprecation_folder = RULES_CONFIG.rule_dirs[0] / '_deprecated' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
better to be explicit than implicit - with no guarantee of ordering, it seems somewhat random. This could even change internally, if say we added hunting to the 0 index.
if not deprecation_folder: | |
deprecation_folder = RULES_CONFIG.rule_dirs[0] / '_deprecated' |
There are a few alternative better options:
- you can use click callbacks (or default as a function) to make the default
rule.get_base_rule_dir() / '_deprecated' / rule_file.name
if no custom rules dir envvar is set otherwiserequire
- update the config to explicitly set a deprecated folder (with comments that its not needed in some instances)
- just make it an argument or required option always, where even we must explicitly call the deprecated dir
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be done in another PR along with the other examples DEFAULT_PREBUILT_RULES_DIRS[0]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid this ambiguous situation, we decided the best course of action at this point is to go with the 3rd option and just make it a required click option - enforcing it to always be explicit (including for us)
I left a few final comments for review. Beyond that, I think the last remaining For the sake of progress, we can address it in a future PR, but I think we should def revisit it before declaring alpha |
++ Concur I think any more ideas should be migrated to another PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We were able to discuss the remaining two items in depth (albeit, maybe too long and in depth 😅 )
1. Deprecated folder
To avoid the ambiguity and introducing an ambiguous intermediate state, the code changes suggested changes the deprecated folder to always be explicitly passed
2. latest_version
and autobumped_version
After hours of discussion, we realized that, beyond just my concern of autobumped_version
functionally changing too far, that implicitly returning 1 without control was also problematic.
We took advantage of this to add more clarity, since the methods and constructs as is were developed and exist in a state where the only versioning concept is version lock. To that end, the summary of requested changes:
- rename
latest_version
tosaved_version
, clarify in the docstring that it pulls saved versions only, with precedence to toml file (over version lock), whereNone
will be returned if not present- a HUGE advantage of this change is it now gives us a way to differentiate between defer to kibana and define in rule toml (previously we could only determine bypass or not). This opens up opportunity for tightening down in future PRs (such as, within the rule loader, enforcing a consistent state)
- update
autobumped_version
to raise the error whenBYPASS_VERSION_LOCK
since it will never be valid - expose a new convenience method
get_synthetic_version
with control over how to handle missing (1 orNone
)- there are 8 existing calls using
autobumped_version
which will need to be changed, but an issue can be made for that to happen in the next PR
- there are 8 existing calls using
Github rendering the diffs is a bit ugly and may be making this less readable, so here is the flattened diff of the 2 changes to help clarify
diff --git a/detection_rules/devtools.py b/detection_rules/devtools.py
index c0b0a1150..386b7dc20 100644
--- a/detection_rules/devtools.py
+++ b/detection_rules/devtools.py
@@ -739,7 +739,8 @@ def search_rule_prs(ctx, no_loop, query, columns, language, token, threads):
@dev_group.command('deprecate-rule')
@click.argument('rule-file', type=Path)
-@click.option('--deprecation-folder', '-d', type=Path)
+@click.option('--deprecation-folder', '-d', type=Path, required=True,
+ help='Location to move the deprecated rule file to')
@click.pass_context
def deprecate_rule(ctx: click.Context, rule_file: Path, deprecation_folder: Path):
"""Deprecate a rule."""
@@ -754,9 +755,6 @@ def deprecate_rule(ctx: click.Context, rule_file: Path, deprecation_folder: Path
ctx.exit()
today = time.strftime('%Y/%m/%d')
- if not deprecation_folder:
- deprecation_folder = RULES_CONFIG.rule_dirs[0] / '_deprecated'
-
deprecated_path = deprecation_folder / rule_file.name
# create the new rule and save it
diff --git a/detection_rules/packaging.py b/detection_rules/packaging.py
index 050af8b9e..bc0884f0c 100644
--- a/detection_rules/packaging.py
+++ b/detection_rules/packaging.py
@@ -92,10 +92,10 @@ class Package(object):
self.historical = historical
if min_version is not None:
- self.rules = self.rules.filter(lambda r: min_version <= r.contents.latest_version)
+ self.rules = self.rules.filter(lambda r: min_version <= r.contents.saved_version)
if max_version is not None:
- self.rules = self.rules.filter(lambda r: max_version >= r.contents.latest_version)
+ self.rules = self.rules.filter(lambda r: max_version >= r.contents.saved_version)
assert not RULES_CONFIG.bypass_version_lock, "Packaging can not be used when version locking is bypassed."
self.changed_ids, self.new_ids, self.removed_ids = \
diff --git a/detection_rules/rule.py b/detection_rules/rule.py
index 973bf5a9f..bac68c718 100644
--- a/detection_rules/rule.py
+++ b/detection_rules/rule.py
@@ -937,7 +937,7 @@ class BaseRuleContents(ABC):
pass
def lock_info(self, bump=True) -> dict:
- version = self.autobumped_version if bump else (self.latest_version or 1)
+ version = self.autobumped_version if bump else (self.saved_version or 1)
contents = {"rule_name": self.name, "sha256": self.sha256(), "version": version, "type": self.type}
return contents
@@ -984,12 +984,12 @@ class BaseRuleContents(ABC):
return max_allowable_version - current_version - 1
@property
- def latest_version(self) -> Optional[int]:
- """Retrieve the latest known version of the rule, considering the version lock setting."""
+ def saved_version(self) -> Optional[int]:
+ """Retrieve the version from the version.lock or from the file if version locking is bypassed."""
toml_version = self.data.get("version")
if BYPASS_VERSION_LOCK:
- return toml_version or 1
+ return toml_version
if toml_version:
print(f"WARNING: Rule {self.name} - {self.id} has a version set in the rule TOML."
@@ -1001,10 +1001,10 @@ class BaseRuleContents(ABC):
@property
def autobumped_version(self) -> Optional[int]:
"""Retrieve the current version of the rule, accounting for automatic increments."""
- version = self.latest_version
+ version = self.saved_version
if BYPASS_VERSION_LOCK:
- return version
+ raise NotImplementedError("This method is not implemented when version locking is not in use.")
# Default to version 1 if no version is set yet
if version is None:
@@ -1013,6 +1013,14 @@ class BaseRuleContents(ABC):
# Auto-increment version if the rule is 'dirty' and not bypassing version lock
return version + 1 if self.is_dirty else version
+ def get_synthetic_version(self, use_default: bool) -> Optional[int]:
+ """
+ Get the latest actual representation of a rule's version, where changes are accounted for automatically when
+ version locking is used, otherwise, return the version defined in the rule toml if present else optionally
+ default to 1.
+ """
+ return self.autobumped_version or self.saved_version or (1 if use_default else None)
+
@classmethod
def convert_supported_version(cls, stack_version: Optional[str]) -> Version:
"""Convert an optional stack version to the minimum for the lock in the form major.minor."""
diff --git a/detection_rules/version_lock.py b/detection_rules/version_lock.py
index 43d7c1cac..b48515f1c 100644
--- a/detection_rules/version_lock.py
+++ b/detection_rules/version_lock.py
@@ -189,7 +189,7 @@ class VersionLock:
already_deprecated = set(current_deprecated_lock)
deprecated_rules = set(rules.deprecated.id_map)
- new_rules = set(rule.id for rule in rules if rule.contents.latest_version is None) - deprecated_rules
+ new_rules = set(rule.id for rule in rules if rule.contents.saved_version is None) - deprecated_rules
changed_rules = set(rule.id for rule in rules if rule.contents.is_dirty) - deprecated_rules
# manage deprecated rules
If it is easier for me to just push these unstaged changes, let me know
After these 2, this LGTM
detection_rules/rule.py
Outdated
@@ -966,17 +985,32 @@ def get_version_space(self) -> Optional[int]: | |||
|
|||
@property | |||
def latest_version(self) -> Optional[int]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def latest_version(self) -> Optional[int]: | |
def saved_version(self) -> Optional[int]: |
detection_rules/rule.py
Outdated
"""Retrieve the latest known version of the rule.""" | ||
min_stack = self.get_supported_version() | ||
return self.version_lock.get_locked_version(self.id, min_stack) | ||
"""Retrieve the latest known version of the rule, considering the version lock setting.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"""Retrieve the latest known version of the rule, considering the version lock setting.""" | |
"""Retrieve the version from the version.lock or from the file if version locking is bypassed.""" |
detection_rules/rule.py
Outdated
toml_version = self.data.get("version") | ||
|
||
if BYPASS_VERSION_LOCK: | ||
return toml_version or 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return toml_version or 1 | |
return toml_version |
detection_rules/rule.py
Outdated
|
||
@property | ||
def autobumped_version(self) -> Optional[int]: | ||
"""Retrieve the current version of the rule, accounting for automatic increments.""" | ||
version = self.latest_version | ||
|
||
if BYPASS_VERSION_LOCK: | ||
return version |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return version | |
raise NotImplementedError("This method is not implemented when version locking is not in use.") |
if version is None: | ||
return 1 | ||
|
||
# Auto-increment version if the rule is 'dirty' and not bypassing version lock | ||
return version + 1 if self.is_dirty else version | ||
|
||
@classmethod |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@classmethod | |
def get_synthetic_version(self, use_default: bool) -> Optional[int]: | |
""" | |
Get the latest actual representation of a rule's version, where changes are accounted for automatically when | |
version locking is used, otherwise, return the version defined in the rule toml if present else optionally | |
default to 1. | |
""" | |
return self.autobumped_version or self.saved_version or (1 if use_default else None) | |
@classmethod |
detection_rules/devtools.py
Outdated
if not deprecation_folder: | ||
deprecation_folder = RULES_CONFIG.rule_dirs[0] / '_deprecated' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid this ambiguous situation, we decided the best course of action at this point is to go with the 3rd option and just make it a required click option - enforcing it to always be explicit (including for us)
detection_rules/devtools.py
Outdated
@@ -721,21 +739,25 @@ def add_github_meta(this_rule: TOMLRule, status: str, original_rule_id: Optional | |||
|
|||
@dev_group.command('deprecate-rule') | |||
@click.argument('rule-file', type=Path) | |||
@click.option('--deprecation-folder', '-d', type=Path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@click.option('--deprecation-folder', '-d', type=Path) | |
@click.option('--deprecation-folder', '-d', type=Path, required=True, help='Location to move the rule file to') |
Applied the above patch 🚀 |
Note unit test failure not related to the code change. Locally works fine (I think it may be a dependency that needs upgrading or something) Re-added in latest commit. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considerable Peer review, LGTM 🙂 🟢
Issues
Resolves #3620
Summary
Adds limited support to validate use of the
version
orrevision
(added in 8.8) field within the TOML file. This approach leverages the toml path and simply checks if it's in a prebuilt rule directory, and raises an error if these fields are present in prebuilt rules. We could alternatively pass the path down and do validation earlier.Bug Fix
../../
and adds aresolve()
to complete the path. (this way the RulesConfig works for the custom dir and prebuilt rule directories)Testing
Here are a series of commands used to test different aspects of the CLI:
Testing Prebuilt Rules
...
A: a577e524-c2ee-47bd-9c5b-e917d01d3276, new version: 2
- min_stack_version added: 8.11.0
A: 947827c6-9ed6-4dec-903e-c856c86e72f3, new version: 3
- min_stack_version added: 8.3.0
A: f94e898e-94f1-4545-8923-03e4b2866211, new version: 1
- min_stack_version added: 8.8.0
A: 61336fe6-c043-4743-ab6e-41292f439603, new version: 1
- min_stack_version added: 8.3.0
A: ee53d67a-5f0c-423c-a53c-8084ae562b5c, new version: 2
- min_stack_version added: 8.3.0
A: 846fe13f-6772-4c83-bd39-9d16d4ad1a81, new version: 5
- min_stack_version added: 8.3.0
A: cf6995ec-32a9-4b2d-9340-f8e61acf3f4e, new version: 2
- min_stack_version added: 8.3.0
Updated /Users/stryker/workspace/Elastic/detection-rules/detection_rules/etc/version.lock.json file
Package saved to: /Users/stryker/workspace/Elastic/detection-rules/releases/8.15
loaded security_detection_engine manifests from the following package versions: ['8.14.1', '8.13.7', '8.13.6', '8.13.5', '8.13.4', '8.13.3', '8.13.2', '8.13.1', '8.12.12', '8.12.11', '8.12.10', '8.12.9', '8.12.8', '8.12.7', '8.12.6', '8.12.5', '8.12.4', '8.12.3', '8.12.2', '8.12.1', '8.11.16', '8.11.15', '8.11.14', '8.11.13', '8.11.12', '8.11.11', '8.11.10', '8.11.9', '8.11.8', '8.11.7', '8.11.6', '8.11.5', '8.11.4', '8.11.3', '8.11.2', '8.11.1', '8.10.18', '8.10.17', '8.10.16', '8.10.15', '8.10.14', '8.10.13', '8.10.12', '8.10.11', '8.10.10', '8.10.9', '8.10.8', '8.10.7', '8.10.6', '8.10.5', '8.10.4', '8.10.3', '8.10.2', '8.10.1', '8.9.15', '8.9.14', '8.9.13', '8.9.12', '8.9.11', '8.9.10', '8.9.9', '8.9.8', '8.9.7', '8.9.6', '8.9.5', '8.9.4', '8.9.3', '8.9.2', '8.9.1', '8.8.15', '8.8.14', '8.8.13', '8.8.12', '8.8.11', '8.8.10', '8.8.9', '8.8.8', '8.8.7', '8.8.6', '8.8.5', '8.8.4', '8.8.3', '8.8.2', '8.8.1', '8.7.13', '8.7.12', '8.7.11', '8.7.10', '8.7.9', '8.7.8', '8.7.7', '8.7.6', '8.7.5', '8.7.4', '8.7.3', '8.7.2', '8.7.1', '8.6.10', '8.6.9', '8.6.8', '8.6.7', '8.6.6', '8.6.5', '8.6.4', '8.6.3', '8.6.2', '8.6.1', '8.5.8', '8.5.7', '8.5.6', '8.5.5', '8.5.4', '8.5.3', '8.5.2', '8.5.1', '8.4.5', '8.4.4', '8.4.3', '8.4.2', '8.4.1', '8.3.4', '8.3.3', '8.3.2', '8.3.1', '8.2.1', '8.1.1', '1.0.2', '1.0.1']
[+] Adding historical rules from 8.14.1 package
sha256: b7c047a58f9a3acc01432dcff6bdffe188974b9face76d99364e2adf034136fb
1108 rules included
python -m detection_rules dev deprecate-rule /Users/stryker/workspace/Elastic/detection-rules/rules/linux/command_and_control_cat_network_activity.toml
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Rule moved to /Users/stryker/workspace/Elastic/detection-rules/rules/linux/_deprecated/command_and_control_cat_network_activity.toml - remember to git add this file
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Are you sure you want to update hashes for 1107 rules without a version bump? [y/N]: y
No hashes updated
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
skipping Enumeration of Privileged Local Groups Membership - ValidationError
skipping test new terms rule - ValidationError
skipping test investigation_fields - ValidationError
skipping Suppression is not enabled, threshold fields not selected - KqlParseError
skipping Suppression is enabled, threshold fields selected - KqlParseError
skipping test rule - ValidationError
skipping test rule - ValidationError
skipping test2 - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping Potential Malicious File Downloaded from Google Drive - ValidationError
skipping Exchange Mailbox Export via PowerShell - ValidationError
skipping ertewr - ValidationError
19 rules exported
4 rules converted
4 saved to mikas_folder
15 errors saved to mikas_folder/_errors.txt
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
python -m detection_rules kibana import-rules -o -d mikas_folder/rules
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
1 rule(s) successfully imported
3bd3801f-33ca-42ec-8147-08bbc94c2f09
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
python -m detection_rules import-rules-to-repo /Users/stryker/Downloads/rules_export-8.ndjson -s rules --required-only
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[+] Building rule for rules/potential_malicious_file_downloaded_from_google_drive.toml
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Exported 1113 rules into /Users/stryker/workspace/Elastic/detection-rules/exports/20240520T071312L.ndjson
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Testing Custom Rules W/ bypass_version_lock: False
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Rule moved to /Users/stryker/workspace/Elastic/detection-rules/mikas_folder/rules/windows/_deprecated/potential_malicious_file_downloaded_from_google_drive.toml - remember to git add this file
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Are you sure you want to update hashes for 1 rules without a version bump? [y/N]: y
WARNING: Rule Potential Malicious File Downloaded from Google Drive - 30b0e855-6ebc-457e-85ed-c33e52d0c6fc has a version set in the rule TOML. This
version
will be ignored and defaulted to the version.lock.json file. Setbypass_version_lock
toTrue
in the rules config to use the TOML version.No hashes updated
python -m detection_rules dev trim-version-lock 8.9.0
python -m detection_rules kibana export-rules -d mikas_folder -s
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
skipping Enumeration of Privileged Local Groups Membership - ValidationError
skipping test new terms rule - ValidationError
skipping test investigation_fields - ValidationError
skipping Suppression is not enabled, threshold fields not selected - KqlParseError
skipping Suppression is enabled, threshold fields selected - KqlParseError
skipping test rule - ValidationError
skipping test rule - ValidationError
skipping test2 - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping Potential Malicious File Downloaded from Google Drive - ValidationError
skipping Potential Malicious File Downloaded from Google Drive - ValidationError
skipping Exchange Mailbox Export via PowerShell - ValidationError
skipping ertewr - ValidationError
19 rules exported
3 rules converted
3 saved to mikas_folder
16 errors saved to mikas_folder/_errors.txt
python -m detection_rules kibana import-rules -o -d mikas_folder/rules
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
WARNING: Rule Potential Malicious File Downloaded from Google Drive - 30b0e855-6ebc-457e-85ed-c33e52d0c6fc has a version set in the rule TOML. This
version
will be ignored and defaulted to the version.lock.json file. Setbypass_version_lock
toTrue
in the rules config to use the TOML version.1 rule(s) successfully imported
30b0e855-6ebc-457e-85ed-c33e52d0c6fc
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
python -m detection_rules import-rules-to-repo /Users/stryker/Downloads/rules_export-8.ndjson -s mikas_folder/rules --required-only
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[+] Building rule for mikas_folder/rules/potential_malicious_file_downloaded_from_google_drive.toml
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Exported 1 rules into /Users/stryker/workspace/Elastic/detection-rules/exports/20240520T064059L.ndjson
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
{
"author": [
"Elastic"
],
"description": "Identifies potential malicious file download and execution from Google Drive. The rule checks for download activity from Google Drive URL, followed by the creation of files commonly leveraged by or for malware. This could indicate an attempt to run malicious scripts, executables or payloads.",
"index": [
"auditbeat-",
"logs-endpoint"
],
"language": "eql",
"name": "Potential Malicious File Downloaded from Google Drive",
"query": "sequence by host.id, process.entity_id with maxspan=30s\n[any where\n\n /* Look for processes started or libraries loaded from untrusted or unsigned Windows, Linux or macOS binaries /\n (event.action in ("exec", "fork", "start", "load")) or\n\n / Look for Google Drive download URL with AV flag skipping /\n (process.args : "drive.google.com" and process.args : "export=download" and process.args : "confirm=no_antivirus")\n]\n\n[network where\n / Look for DNS requests for Google Drive /\n (dns.question.name : "drive.google.com" and dns.question.type : "A") or\n\n / Look for connection attempts to address that resolves to Google /\n (destination.as.organization.name : "GOOGLE" and event.action == "connection_attempted")\n\n / NOTE: Add LoLBins if tuning is required\n process.name : (\n "cmd.exe", "bitsadmin.exe", "certutil.exe", "esentutl.exe", "wmic.exe", "PowerShell.exe",\n "homedrive.exe","regsvr32.exe", "mshta.exe", "rundll32.exe", "cscript.exe", "wscript.exe",\n "curl", "wget", "scp", "ftp", "python", "perl", "ruby"))] /\n]\n\n/ Identify the creation of files following Google Drive connection with extensions commonly used for executables or libraries */\n[file where event.action == "creation" and file.extension : (\n "exe", "dll", "scr", "jar", "pif", "app", "dmg", "pkg", "elf", "so", "bin", "deb", "rpm","sh","hta","lnk"\n )\n]\n",
"required_fields": [
{
"ecs": true,
"name": "destination.as.organization.name",
"type": "keyword"
},
{
"ecs": true,
"name": "dns.question.name",
"type": "keyword"
},
{
"ecs": true,
"name": "dns.question.type",
"type": "keyword"
},
{
"ecs": true,
"name": "event.action",
"type": "keyword"
},
{
"ecs": true,
"name": "file.extension",
"type": "keyword"
},
{
"ecs": true,
"name": "host.id",
"type": "keyword"
},
{
"ecs": true,
"name": "process.args",
"type": "keyword"
},
{
"ecs": true,
"name": "process.entity_id",
"type": "keyword"
}
],
"risk_score": 73,
"rule_id": "3bd3801f-33ca-42ec-8147-08bbc94c2f09",
"severity": "high",
"type": "eql",
"version": 1
}
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[metadata]
creation_date = "2024/05/20"
maturity = "development"
updated_date = "2024/05/20"
[rule]
author = ["Elastic"]
description = """
Identifies potential malicious file download and execution from Google Drive. The rule checks for download activity from
Google Drive URL, followed by the creation of files commonly leveraged by or for malware. This could indicate an attempt
to run malicious scripts, executables or payloads.
"""
index = ["auditbeat-", "logs-endpoint"]
language = "eql"
name = "Potential Malicious File Downloaded from Google Drive"
risk_score = 73
rule_id = "3bd3801f-33ca-42ec-8147-08bbc94c2f09"
severity = "high"
type = "eql"
query = '''
sequence by host.id, process.entity_id with maxspan=30s
[any where
]
[network where
/* Look for DNS requests for Google Drive */
(dns.question.name : "drive.google.com" and dns.question.type : "A") or
]
/* Identify the creation of files following Google Drive connection with extensions commonly used for executables or libraries */
[file where event.action == "creation" and file.extension : (
"exe", "dll", "scr", "jar", "pif", "app", "dmg", "pkg", "elf", "so", "bin", "deb", "rpm","sh","hta","lnk"
)
]
'''
Testing Custom Rules W/ bypass_version_lock: True
- python -m detection_rules dev build-release --update-version-lock Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[+] Building package 8.14
Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.1 rules excluded from package
WARNING: Cannot bypass version lock. Version lock is enabled. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.WARNING: Version Lock usage is bypassed. Set
bypass_version_lock: false
in the rules config to enable.WARNING: You cannot update the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.python -m detection_rules dev deprecate-rule /Users/stryker/workspace/Elastic/detection-rules/mikas_folder/rules/potential_malicious_file_downloaded_from_google_drive.toml
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot deprecate a rule when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.Are you sure you want to update hashes for 0 rules without a version bump? [y/N]: y
WARNING: You cannot run this command when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
toFalse
in the rules config to use the version lock.(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot trim the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
skipping Enumeration of Privileged Local Groups Membership - ValidationError
skipping test new terms rule - ValidationError
skipping test investigation_fields - ValidationError
skipping Suppression is not enabled, threshold fields not selected - KqlParseError
skipping Suppression is enabled, threshold fields selected - KqlParseError
skipping test rule - ValidationError
skipping test rule - ValidationError
skipping test2 - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping First Time Seen AWS Secret Value Accessed in Secrets Manager - ValidationError
skipping Potential Malicious File Downloaded from Google Drive - ValidationError
skipping Exchange Mailbox Export via PowerShell - ValidationError
skipping ertewr - ValidationError
19 rules exported
4 rules converted
4 saved to mikas_folder
15 errors saved to mikas_folder/_errors.txt
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
python -m detection_rules kibana import-rules -o -d mikas_folder/rules
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.1 rule(s) successfully imported
3bd3801f-33ca-42ec-8147-08bbc94c2f09
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
python -m detection_rules import-rules-to-repo /Users/stryker/Downloads/rules_export-8.ndjson -s mikas_folder/rules --required-only
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
[+] Building rule for mikas_folder/rules/potential_malicious_file_downloaded_from_google_drive.toml
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.Exported 1 rules into /Users/stryker/workspace/Elastic/detection-rules/exports/20240520T070113L.ndjson
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.{
"author": [
"Elastic"
],
"description": "Identifies potential malicious file download and execution from Google Drive. The rule checks for download activity from Google Drive URL, followed by the creation of files commonly leveraged by or for malware. This could indicate an attempt to run malicious scripts, executables or payloads.",
"index": [
"auditbeat-",
"logs-endpoint"
],
"language": "eql",
"name": "Potential Malicious File Downloaded from Google Drive",
"query": "sequence by host.id, process.entity_id with maxspan=30s\n[any where\n\n /* Look for processes started or libraries loaded from untrusted or unsigned Windows, Linux or macOS binaries /\n (event.action in ("exec", "fork", "start", "load")) or\n\n / Look for Google Drive download URL with AV flag skipping /\n (process.args : "drive.google.com" and process.args : "export=download" and process.args : "confirm=no_antivirus")\n]\n\n[network where\n / Look for DNS requests for Google Drive /\n (dns.question.name : "drive.google.com" and dns.question.type : "A") or\n\n / Look for connection attempts to address that resolves to Google /\n (destination.as.organization.name : "GOOGLE" and event.action == "connection_attempted")\n\n / NOTE: Add LoLBins if tuning is required\n process.name : (\n "cmd.exe", "bitsadmin.exe", "certutil.exe", "esentutl.exe", "wmic.exe", "PowerShell.exe",\n "homedrive.exe","regsvr32.exe", "mshta.exe", "rundll32.exe", "cscript.exe", "wscript.exe",\n "curl", "wget", "scp", "ftp", "python", "perl", "ruby"))] /\n]\n\n/ Identify the creation of files following Google Drive connection with extensions commonly used for executables or libraries */\n[file where event.action == "creation" and file.extension : (\n "exe", "dll", "scr", "jar", "pif", "app", "dmg", "pkg", "elf", "so", "bin", "deb", "rpm","sh","hta","lnk"\n )\n]\n",
"required_fields": [
{
"ecs": true,
"name": "destination.as.organization.name",
"type": "keyword"
},
{
"ecs": true,
"name": "dns.question.name",
"type": "keyword"
},
{
"ecs": true,
"name": "dns.question.type",
"type": "keyword"
},
{
"ecs": true,
"name": "event.action",
"type": "keyword"
},
{
"ecs": true,
"name": "file.extension",
"type": "keyword"
},
{
"ecs": true,
"name": "host.id",
"type": "keyword"
},
{
"ecs": true,
"name": "process.args",
"type": "keyword"
},
{
"ecs": true,
"name": "process.entity_id",
"type": "keyword"
}
],
"risk_score": 73,
"rule_id": "3bd3801f-33ca-42ec-8147-08bbc94c2f09",
"severity": "high",
"type": "eql",
"version": 1
}
(detection-rules-build) ➜ detection-rules git:(3620-frdac-add-limited-support-for-version-and-revision-to-baseruledata) ✗
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json
█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄ ▄ █▀▀▄ ▄ ▄ ▄ ▄▄▄ ▄▄▄
█ █ █▄▄ █ █▄▄ █ █ █ █ █ █▀▄ █ █▄▄▀ █ █ █ █▄▄ █▄▄
█▄▄▀ █▄▄ █ █▄▄ █▄▄ █ ▄█▄ █▄█ █ ▀▄█ █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
Cannot set the version lock when the versioning strategy is configured to bypass the version lock. Set
bypass_version_lock
tofalse
in the rules config to use the version lock.[metadata]
creation_date = "2024/05/20"
maturity = "development"
updated_date = "2024/05/20"
[rule]
author = ["Elastic"]
description = """
Identifies potential malicious file download and execution from Google Drive. The rule checks for download activity from
Google Drive URL, followed by the creation of files commonly leveraged by or for malware. This could indicate an attempt
to run malicious scripts, executables or payloads.
"""
index = ["auditbeat-", "logs-endpoint"]
language = "eql"
name = "Potential Malicious File Downloaded from Google Drive"
risk_score = 73
rule_id = "3bd3801f-33ca-42ec-8147-08bbc94c2f09"
severity = "high"
type = "eql"
query = '''
sequence by host.id, process.entity_id with maxspan=30s
[any where
]
[network where
/* Look for DNS requests for Google Drive */
(dns.question.name : "drive.google.com" and dns.question.type : "A") or
]
/* Identify the creation of files following Google Drive connection with extensions commonly used for executables or libraries */
[file where event.action == "creation" and file.extension : (
"exe", "dll", "scr", "jar", "pif", "app", "dmg", "pkg", "elf", "so", "bin", "deb", "rpm","sh","hta","lnk"
)
]
'''
Version Strategy Warning
bypass_version_lock = False
)bypass_version_lock = True
)bypass_version_lock = False
. No warning message is issued.bypass_version_lock = True
when building a release package (build_release). A warning message is issued.bypass_version_lock = False
, the versions in the TOML file will not be used (autobumped_version). A warning message is issued.bypass_version_lock = False
, when autobumping the version, it will check the version lock file and increment if is_dirty (autobumped_version), otherwise just use the version supplied. No warning message is issued.bypass_version_lock = True
, the updating the version lock file will disabled (update_lock_versions). A warning message is issued.bypass_version_lock = True
, loading the version lock file is disabled and skipped. (from_dict, load_from_file, manage_versions, test_version_lock_has_nested_previous). A warning message is issued.