[FR] Improve DAC custom folder init #3653

Mikaayenson · 2024-05-07T15:15:15Z

Issues

Resolves #3621

Summary

Improves the overall experience for users trying to initialize a custom folder using the python -m detection_rules custom-rules init-config command.

init-config --> setup-config: Not just base file structure, but setup with initial file content so users can easily get to testing, and removed the config details since we're not defaulting to initialize files with content. Added a cli param to allow users to specify kibana version during setup. Also echoed doc links post setup.
SystemExit: Decided to keep this raised exit since the RULES_CONFIG = parse_rules_config() is initialized in so many places in the repo. Also because we dont have CLI context to try/catch.
Moved _config.yml validation to a dataclass (note: couldn't use mixins because of the multiple circular dependancies)
Added packages.yaml validation to the post init of the TestConfig
Updated doc references as needed.

A few bug fixes:

The building block folder was missing from the original custom init
Corrected the example configs to match the paths generated by the init

Testing

Generated _config.yaml files

With defaults:

files:
  deprecated_rules: etc/deprecated_rules.json
  packages: etc/packages.yml
  stack_schema_map: etc/stack-schema-map.yaml
  version_lock: etc/version.lock.json
rule_dirs:
- rules
- rules_building_block
testing:
  config: etc/example_test_config.yaml

package:
  name: '8.14'

8.14.0:
  beats: 8.12.2
  ecs: 8.11.0
  endgame: 8.4.0

Testing

(detection-rules-build) ➜  detection-rules git:(3621-frdac-raise-a-better-exception-for-missing-contents-in-config-files) ✗ python -m detection_rules custom-rules setup-config mikas_folder 8.12
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

Created directory: mikas_folder/actions
Created directory: mikas_folder/exceptions
Created directory: mikas_folder/rules
Created directory: mikas_folder/rules_building_block
Created directory: mikas_folder/etc
Created file with default content: mikas_folder/etc/deprecated_rules.json
Created file with default content: mikas_folder/etc/version.lock.json
Created file with default content: mikas_folder/etc/packages.yml
Created file with default content: mikas_folder/etc/stack-schema-map.yaml
Created file with default content: mikas_folder/etc/test_config.yaml
Created file with default content: mikas_folder/_config.yaml

# For details on how to configure the _config.yaml file,
# consult: /Users/stryker/workspace/Elastic/detection-rules/detection_rules/etc/_config.yaml
# or the docs: /Users/stryker/workspace/Elastic/detection-rules/docs/custom-rules.md
(detection-rules-build) ➜  detection-rules git:(3621-frdac-raise-a-better-exception-for-missing-contents-in-config-files) ✗ python -m detection_rules custom-rules setup-config mikas_folder     
Loaded config file: /Users/stryker/workspace/Elastic/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

Created directory: mikas_folder/actions
Created directory: mikas_folder/exceptions
Created directory: mikas_folder/rules
Created directory: mikas_folder/rules_building_block
Created directory: mikas_folder/etc
Created file with default content: mikas_folder/etc/deprecated_rules.json
Created file with default content: mikas_folder/etc/version.lock.json
Created file with default content: mikas_folder/etc/packages.yml
Created file with default content: mikas_folder/etc/stack-schema-map.yaml
Created file with default content: mikas_folder/etc/test_config.yaml
Created file with default content: mikas_folder/_config.yaml

# For details on how to configure the _config.yaml file,
# consult: /Users/stryker/workspace/Elastic/detection-rules/detection_rules/etc/_config.yaml
# or the docs: /Users/stryker/workspace/Elastic/detection-rules/docs/custom-rules.md

Make tests should still pass.

detection_rules/custom_rules.py

detection_rules/config.py

detection_rules/custom_rules.py

detection_rules/main.py

brokensound77 · 2024-05-07T15:35:27Z

In your testing output, can you also cat the _config file for both

detection_rules/custom_rules.py

Mikaayenson · 2024-05-07T16:57:59Z

In your testing output, can you also cat the _config file for both

Updated the summary.

eric-forte-elastic

🟢 Manual review, looks good to me! 👍

Testing 🟢

…ion-rules on  3621-frdac-raise-a-better-exception-for-missing-contents-in-config-files [$] is  v0.1.0 via  v3.12.3 (detection-rules-build) on  eric.forte 
❯ python -m detection_rules custom-rules init-config erics_folder

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

Created directory: erics_folder/actions
Created directory: erics_folder/exceptions
Created directory: erics_folder/rules
Created directory: erics_folder/rules_building_block
Created directory: erics_folder/etc
Created file: erics_folder/etc/deprecated_rules.json
Created file: erics_folder/etc/packages.yml
Created file: erics_folder/etc/stack-schema-map.yaml
Created file: erics_folder/etc/version.lock.json
Created file: erics_folder/etc/test_config.yaml
Configured _config.yaml with detailed instructions

…on-rules on  3621-frdac-raise-a-better-exception-for-missing-contents-in-config-files [$?] is  v0.1.0 via  v3.12.3 (detection-rules-build) on  eric.forte 
❯ make test
Installing kql and kibana packages...
./env/detection-rules-build/bin/pip install lib/kql lib/kibana
Looking in indexes: https://pypi.org/simple, https://eric.forte%40elastic.co:****@artifactory.elastic.dev/artifactory/api/pypi/pypi-endgame/simple
Processing ./lib/kql
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Processing ./lib/kibana
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: eql==0.9.19 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection-rules-kql==0.1.7) (0.9.19)
Requirement already satisfied: lark-parser>=0.12.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection-rules-kql==0.1.7) (0.12.0)
Requirement already satisfied: requests<3.0,>=2.25 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection-rules-kibana==0.2.0) (2.31.0)
Requirement already satisfied: elasticsearch~=8.12.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection-rules-kibana==0.2.0) (8.12.1)
Requirement already satisfied: elastic-transport<9,>=8 in ./env/detection-rules-build/lib/python3.12/site-packages (from elasticsearch~=8.12.1->detection-rules-kibana==0.2.0) (8.12.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests<3.0,>=2.25->detection-rules-kibana==0.2.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests<3.0,>=2.25->detection-rules-kibana==0.2.0) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests<3.0,>=2.25->detection-rules-kibana==0.2.0) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests<3.0,>=2.25->detection-rules-kibana==0.2.0) (2024.2.2)
Building wheels for collected packages: detection-rules-kql, detection-rules-kibana
  Building wheel for detection-rules-kql (pyproject.toml) ... done
  Created wheel for detection-rules-kql: filename=detection_rules_kql-0.1.7-py3-none-any.whl size=16336 sha256=b16342078afe841fa8e9d5a956d41610a7a84bf7ebe337b72b1094b56634b59f
  Stored in directory: /tmp/pip-ephem-wheel-cache-aq1z54ig/wheels/56/d8/16/54016845ffe39458d941c5c2dbc03e4998264eeec10328eebb
  Building wheel for detection-rules-kibana (pyproject.toml) ... done
  Created wheel for detection-rules-kibana: filename=detection_rules_kibana-0.2.0-py3-none-any.whl size=9325 sha256=775ecf84396d2ffce3a8815fe8ed048bbe0b2b23c2ea9575f1e7a2c772406fcd
  Stored in directory: /tmp/pip-ephem-wheel-cache-aq1z54ig/wheels/05/3b/89/3dab4aeb78501459dd86df8d126afb94aab37d28aea1263c1d
Successfully built detection-rules-kql detection-rules-kibana
Installing collected packages: detection-rules-kql, detection-rules-kibana
  Attempting uninstall: detection-rules-kql
    Found existing installation: detection-rules-kql 0.1.8
    Uninstalling detection-rules-kql-0.1.8:
      Successfully uninstalled detection-rules-kql-0.1.8
  Attempting uninstall: detection-rules-kibana
    Found existing installation: detection-rules-kibana 0.1.0
    Uninstalling detection-rules-kibana-0.1.0:
      Successfully uninstalled detection-rules-kibana-0.1.0
Successfully installed detection-rules-kibana-0.2.0 detection-rules-kql-0.1.7
Installing all dependencies...
./env/detection-rules-build/bin/pip install .[dev]
Looking in indexes: https://pypi.org/simple, https://eric.forte%40elastic.co:****@artifactory.elastic.dev/artifactory/api/pypi/pypi-endgame/simple
Processing /home/forteea1/Code/clean_mains/detection-rules
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: Click~=8.1.7 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (8.1.7)
Requirement already satisfied: elasticsearch~=8.12.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (8.12.1)
Requirement already satisfied: eql==0.9.19 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.9.19)
Requirement already satisfied: jsl==0.2.4 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.2.4)
Requirement already satisfied: jsonschema>=4.21.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (4.21.1)
Requirement already satisfied: marko==2.0.3 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (2.0.3)
Requirement already satisfied: marshmallow-dataclass~=8.6.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from marshmallow-dataclass[union]~=8.6.0->detection_rules==0.1.0) (8.6.1)
Requirement already satisfied: marshmallow-jsonschema~=0.13.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.13.0)
Requirement already satisfied: marshmallow-union~=0.1.15 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.1.15.post1)
Requirement already satisfied: marshmallow~=3.21.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (3.21.1)
Requirement already satisfied: pytoml==0.1.21 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.1.21)
Requirement already satisfied: PyYAML~=6.0.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (6.0.1)
Requirement already satisfied: requests~=2.31.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (2.31.0)
Requirement already satisfied: toml==0.10.2 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.10.2)
Requirement already satisfied: typing-inspect==0.9.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.9.0)
Requirement already satisfied: typing-extensions==4.10.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (4.10.0)
Requirement already satisfied: XlsxWriter~=3.2.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (3.2.0)
Requirement already satisfied: semver==3.0.2 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (3.0.2)
Requirement already satisfied: lark-parser~=0.12.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from eql==0.9.19->detection_rules==0.1.0) (0.12.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from typing-inspect==0.9.0->detection_rules==0.1.0) (1.0.0)
Requirement already satisfied: pep8-naming==0.13.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (0.13.0)
Requirement already satisfied: PyGithub==2.2.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (2.2.0)
Requirement already satisfied: flake8==7.0.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (7.0.0)
Requirement already satisfied: pyflakes==3.2.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (3.2.0)
Requirement already satisfied: pytest>=8.1.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (8.1.1)
Requirement already satisfied: pre-commit==3.6.2 in ./env/detection-rules-build/lib/python3.12/site-packages (from detection_rules==0.1.0) (3.6.2)
Requirement already satisfied: mccabe<0.8.0,>=0.7.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from flake8==7.0.0->detection_rules==0.1.0) (0.7.0)
Requirement already satisfied: pycodestyle<2.12.0,>=2.11.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from flake8==7.0.0->detection_rules==0.1.0) (2.11.1)
Requirement already satisfied: cfgv>=2.0.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from pre-commit==3.6.2->detection_rules==0.1.0) (3.4.0)
Requirement already satisfied: identify>=1.0.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from pre-commit==3.6.2->detection_rules==0.1.0) (2.5.35)
Requirement already satisfied: nodeenv>=0.11.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from pre-commit==3.6.2->detection_rules==0.1.0) (1.8.0)
Requirement already satisfied: virtualenv>=20.10.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from pre-commit==3.6.2->detection_rules==0.1.0) (20.25.1)
Requirement already satisfied: pynacl>=1.4.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from PyGithub==2.2.0->detection_rules==0.1.0) (1.5.0)
Requirement already satisfied: pyjwt>=2.4.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.2.0->detection_rules==0.1.0) (2.8.0)
Requirement already satisfied: urllib3>=1.26.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from PyGithub==2.2.0->detection_rules==0.1.0) (2.2.1)
Requirement already satisfied: Deprecated in ./env/detection-rules-build/lib/python3.12/site-packages (from PyGithub==2.2.0->detection_rules==0.1.0) (1.2.14)
Requirement already satisfied: elastic-transport<9,>=8 in ./env/detection-rules-build/lib/python3.12/site-packages (from elasticsearch~=8.12.1->detection_rules==0.1.0) (8.12.0)
Requirement already satisfied: attrs>=22.2.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from jsonschema>=4.21.1->detection_rules==0.1.0) (23.2.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in ./env/detection-rules-build/lib/python3.12/site-packages (from jsonschema>=4.21.1->detection_rules==0.1.0) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in ./env/detection-rules-build/lib/python3.12/site-packages (from jsonschema>=4.21.1->detection_rules==0.1.0) (0.33.0)
Requirement already satisfied: rpds-py>=0.7.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from jsonschema>=4.21.1->detection_rules==0.1.0) (0.18.0)
Requirement already satisfied: packaging>=17.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from marshmallow~=3.21.1->detection_rules==0.1.0) (24.0)
Requirement already satisfied: typeguard<4.0.0,>=2.4.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from marshmallow-dataclass[union]~=8.6.0->detection_rules==0.1.0) (3.0.2)
Requirement already satisfied: iniconfig in ./env/detection-rules-build/lib/python3.12/site-packages (from pytest>=8.1.1->detection_rules==0.1.0) (2.0.0)
Requirement already satisfied: pluggy<2.0,>=1.4 in ./env/detection-rules-build/lib/python3.12/site-packages (from pytest>=8.1.1->detection_rules==0.1.0) (1.4.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests~=2.31.0->detection_rules==0.1.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests~=2.31.0->detection_rules==0.1.0) (3.6)
Requirement already satisfied: certifi>=2017.4.17 in ./env/detection-rules-build/lib/python3.12/site-packages (from requests~=2.31.0->detection_rules==0.1.0) (2024.2.2)
Requirement already satisfied: setuptools in ./env/detection-rules-build/lib/python3.12/site-packages (from nodeenv>=0.11.1->pre-commit==3.6.2->detection_rules==0.1.0) (69.2.0)
Requirement already satisfied: cryptography>=3.4.0 in ./env/detection-rules-build/lib/python3.12/site-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.2.0->detection_rules==0.1.0) (42.0.5)
Requirement already satisfied: cffi>=1.4.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from pynacl>=1.4.0->PyGithub==2.2.0->detection_rules==0.1.0) (1.16.0)
Requirement already satisfied: distlib<1,>=0.3.7 in ./env/detection-rules-build/lib/python3.12/site-packages (from virtualenv>=20.10.0->pre-commit==3.6.2->detection_rules==0.1.0) (0.3.8)
Requirement already satisfied: filelock<4,>=3.12.2 in ./env/detection-rules-build/lib/python3.12/site-packages (from virtualenv>=20.10.0->pre-commit==3.6.2->detection_rules==0.1.0) (3.13.1)
Requirement already satisfied: platformdirs<5,>=3.9.1 in ./env/detection-rules-build/lib/python3.12/site-packages (from virtualenv>=20.10.0->pre-commit==3.6.2->detection_rules==0.1.0) (4.2.0)
Requirement already satisfied: wrapt<2,>=1.10 in ./env/detection-rules-build/lib/python3.12/site-packages (from Deprecated->PyGithub==2.2.0->detection_rules==0.1.0) (1.16.0)
Requirement already satisfied: pycparser in ./env/detection-rules-build/lib/python3.12/site-packages (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.2.0->detection_rules==0.1.0) (2.21)
Building wheels for collected packages: detection_rules
  Building wheel for detection_rules (pyproject.toml) ... done
  Created wheel for detection_rules: filename=detection_rules-0.1.0-py3-none-any.whl size=39609775 sha256=0f2e6ebc4e628f20521bf2adc0e300f7e888490489e0f8fdd6fa896e97edec08
  Stored in directory: /home/forteea1/.cache/pip/wheels/33/0b/6f/442542fc0e808e368b4dfaf768ed0b61a5d8281942974600d4
Successfully built detection_rules
Installing collected packages: detection_rules
  Attempting uninstall: detection_rules
    Found existing installation: detection_rules 0.1.0
    Uninstalling detection_rules-0.1.0:
      Successfully uninstalled detection_rules-0.1.0
Successfully installed detection_rules-0.1.0
LINTING
./env/detection-rules-build/bin/python -m flake8 tests detection_rules --ignore D203,N815 --max-line-length 120
./env/detection-rules-build/bin/python -m detection_rules test

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

Tests skipped per config (144):
tests/kuery/test_dsl.py::TestKQLtoDSL::test_and_query
tests/kuery/test_dsl.py::TestKQLtoDSL::test_field_exists
tests/kuery/test_dsl.py::TestKQLtoDSL::test_field_inequality
tests/kuery/test_dsl.py::TestKQLtoDSL::test_field_match
tests/kuery/test_dsl.py::TestKQLtoDSL::test_not_query
tests/kuery/test_dsl.py::TestKQLtoDSL::test_optimizations
tests/kuery/test_dsl.py::TestKQLtoDSL::test_or_query
tests/kuery/test_eql2kql.py::TestEql2Kql::test_and_query
tests/kuery/test_eql2kql.py::TestEql2Kql::test_boolean_precedence
tests/kuery/test_eql2kql.py::TestEql2Kql::test_field_equals
tests/kuery/test_eql2kql.py::TestEql2Kql::test_field_inequality
tests/kuery/test_eql2kql.py::TestEql2Kql::test_ip_checks
tests/kuery/test_eql2kql.py::TestEql2Kql::test_list_of_values
tests/kuery/test_eql2kql.py::TestEql2Kql::test_not_query
tests/kuery/test_eql2kql.py::TestEql2Kql::test_or_query
tests/kuery/test_eql2kql.py::TestEql2Kql::test_wildcard_field
tests/kuery/test_evaluator.py::EvaluatorTests::test_and_expr
tests/kuery/test_evaluator.py::EvaluatorTests::test_and_values
tests/kuery/test_evaluator.py::EvaluatorTests::test_cidr_match
tests/kuery/test_evaluator.py::EvaluatorTests::test_field_exists
tests/kuery/test_evaluator.py::EvaluatorTests::test_flattening
tests/kuery/test_evaluator.py::EvaluatorTests::test_list_value
tests/kuery/test_evaluator.py::EvaluatorTests::test_not_value
tests/kuery/test_evaluator.py::EvaluatorTests::test_or_expr
tests/kuery/test_evaluator.py::EvaluatorTests::test_or_values
tests/kuery/test_evaluator.py::EvaluatorTests::test_quoted_wildcard
tests/kuery/test_evaluator.py::EvaluatorTests::test_range
tests/kuery/test_evaluator.py::EvaluatorTests::test_single_value
tests/kuery/test_evaluator.py::EvaluatorTests::test_wildcard
tests/kuery/test_kql2eql.py::TestKql2Eql::test_and_query
tests/kuery/test_kql2eql.py::TestKql2Eql::test_boolean_precedence
tests/kuery/test_kql2eql.py::TestKql2Eql::test_field_equals
tests/kuery/test_kql2eql.py::TestKql2Eql::test_field_inequality
tests/kuery/test_kql2eql.py::TestKql2Eql::test_list_of_values
tests/kuery/test_kql2eql.py::TestKql2Eql::test_lone_value
tests/kuery/test_kql2eql.py::TestKql2Eql::test_nested_query
tests/kuery/test_kql2eql.py::TestKql2Eql::test_not_query
tests/kuery/test_kql2eql.py::TestKql2Eql::test_or_query
tests/kuery/test_kql2eql.py::TestKql2Eql::test_schema
tests/kuery/test_lint.py::LintTests::test_and_not
tests/kuery/test_lint.py::LintTests::test_compound
tests/kuery/test_lint.py::LintTests::test_double_negate
tests/kuery/test_lint.py::LintTests::test_extract_not
tests/kuery/test_lint.py::LintTests::test_ip
tests/kuery/test_lint.py::LintTests::test_lint_field
tests/kuery/test_lint.py::LintTests::test_lint_precedence
tests/kuery/test_lint.py::LintTests::test_merge_fields
tests/kuery/test_lint.py::LintTests::test_mixed_demorgans
tests/kuery/test_lint.py::LintTests::test_not_demorgans
tests/kuery/test_lint.py::LintTests::test_not_or
tests/kuery/test_lint.py::LintTests::test_upper_tokens
tests/kuery/test_parser.py::ParserTests::test_conversion
tests/kuery/test_parser.py::ParserTests::test_date
tests/kuery/test_parser.py::ParserTests::test_keyword
tests/kuery/test_parser.py::ParserTests::test_list_equals
tests/kuery/test_parser.py::ParserTests::test_multiple_types_fail
tests/kuery/test_parser.py::ParserTests::test_multiple_types_success
tests/kuery/test_parser.py::ParserTests::test_number_exists
tests/kuery/test_parser.py::ParserTests::test_number_wildcard_fail
tests/kuery/test_parser.py::ParserTests::test_type_family_fail
tests/kuery/test_parser.py::ParserTests::test_type_family_success
tests/test_all_rules.py::TestAlertSuppression::test_group_field_in_schemas
tests/test_all_rules.py::TestBuildTimeFields::test_build_fields_min_stack
tests/test_all_rules.py::TestIncompatibleFields::test_rule_backports_for_restricted_fields
tests/test_all_rules.py::TestIntegrationRules::test_all_min_stack_rules_have_comment
tests/test_all_rules.py::TestIntegrationRules::test_integration_guide
tests/test_all_rules.py::TestIntegrationRules::test_ml_integration_jobs_exist
tests/test_all_rules.py::TestIntegrationRules::test_rule_demotions
tests/test_all_rules.py::TestLicense::test_elastic_license_only_v2
tests/test_all_rules.py::TestNoteMarkdownPlugins::test_if_plugins_explicitly_defined
tests/test_all_rules.py::TestNoteMarkdownPlugins::test_note_has_osquery_warning
tests/test_all_rules.py::TestNoteMarkdownPlugins::test_plugin_placeholders_match_entries
tests/test_all_rules.py::TestRiskScoreMismatch::test_rule_risk_score_severity_mismatch
tests/test_all_rules.py::TestRuleFiles::test_bbr_in_correct_dir
tests/test_all_rules.py::TestRuleFiles::test_non_bbr_in_correct_dir
tests/test_all_rules.py::TestRuleFiles::test_rule_file_name_tactic
tests/test_all_rules.py::TestRuleMetadata::test_deprecated_rules
tests/test_all_rules.py::TestRuleMetadata::test_event_dataset
tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
tests/test_all_rules.py::TestRuleMetadata::test_invalid_queries
tests/test_all_rules.py::TestRuleMetadata::test_updated_date_newer_than_creation
tests/test_all_rules.py::TestRuleTags::test_casing_and_spacing
tests/test_all_rules.py::TestRuleTags::test_investigation_guide_tag
tests/test_all_rules.py::TestRuleTags::test_ml_rule_type_tags
tests/test_all_rules.py::TestRuleTags::test_no_duplicate_tags
tests/test_all_rules.py::TestRuleTags::test_os_tags
tests/test_all_rules.py::TestRuleTags::test_primary_tactic_as_tag
tests/test_all_rules.py::TestRuleTags::test_required_tags
tests/test_all_rules.py::TestRuleTags::test_tag_prefix
tests/test_all_rules.py::TestRuleTimelines::test_timeline_has_title
tests/test_all_rules.py::TestRuleTiming::test_eql_interval_to_maxspan
tests/test_all_rules.py::TestRuleTiming::test_eql_lookback
tests/test_all_rules.py::TestRuleTiming::test_event_override
tests/test_all_rules.py::TestRuleTiming::test_required_lookback
tests/test_all_rules.py::TestThreatMappings::test_duplicated_tactics
tests/test_all_rules.py::TestThreatMappings::test_tactic_to_technique_correlations
tests/test_all_rules.py::TestThreatMappings::test_technique_deprecations
tests/test_all_rules.py::TestValidRules::test_all_rule_queries_optimized
tests/test_all_rules.py::TestValidRules::test_bbr_validation
tests/test_all_rules.py::TestValidRules::test_duplicate_file_names
tests/test_all_rules.py::TestValidRules::test_file_names
tests/test_all_rules.py::TestValidRules::test_production_rules_have_rta
tests/test_all_rules.py::TestValidRules::test_rule_type_changes
tests/test_all_rules.py::TestValidRules::test_schema_and_dupes
tests/test_gh_workflows.py::TestWorkflows::test_matrix_to_lock_version_defaults
tests/test_hunt_data.py::TestHunt::test_load_toml_files
tests/test_hunt_data.py::TestHunt::test_markdown_existence
tests/test_hunt_data.py::TestHunt::test_toml_loading
tests/test_mappings.py::TestMappings::test_false_positives
tests/test_mappings.py::TestMappings::test_true_positives
tests/test_mappings.py::TestRTAs::test_rtas_with_triggered_rules_have_uuid
tests/test_packages.py::TestPackages::test_package_loader_default_configs
tests/test_packages.py::TestPackages::test_package_loader_production_config
tests/test_packages.py::TestPackages::test_package_summary
tests/test_packages.py::TestPackages::test_rule_versioning
tests/test_packages.py::TestRegistryPackage::test_registry_package_config
tests/test_python_library.py::TestEQLInSet::test_eql_in_set
tests/test_schemas.py::TestSchemas::test_eql_validation
tests/test_schemas.py::TestSchemas::test_query_downgrade_7_x
tests/test_schemas.py::TestSchemas::test_query_downgrade_8_x
tests/test_schemas.py::TestSchemas::test_threshold_downgrade_7_x
tests/test_schemas.py::TestSchemas::test_threshold_downgrade_8_x
tests/test_schemas.py::TestSchemas::test_versioned_downgrade_7_x
tests/test_schemas.py::TestSchemas::test_versioned_downgrade_8_x
tests/test_schemas.py::TestVersionLockSchema::test_version_lock_has_nested_previous
tests/test_schemas.py::TestVersionLockSchema::test_version_lock_no_previous
tests/test_schemas.py::TestVersions::test_stack_schema_map
tests/test_specific_rules.py::TestESQLRules::test_esql_queries
tests/test_specific_rules.py::TestEndpointQuery::test_os_and_platform_in_query
tests/test_specific_rules.py::TestNewTerms::test_history_window_start
tests/test_specific_rules.py::TestNewTerms::test_new_terms_field_exists
tests/test_specific_rules.py::TestNewTerms::test_new_terms_fields
tests/test_specific_rules.py::TestNewTerms::test_new_terms_fields_unique
tests/test_specific_rules.py::TestNewTerms::test_new_terms_max_limit
tests/test_toml_formatter.py::TestRuleTomlFormatter::test_formatter_deep
tests/test_toml_formatter.py::TestRuleTomlFormatter::test_formatter_rule
tests/test_toml_formatter.py::TestRuleTomlFormatter::test_normalization
tests/test_transform_fields.py::TestGuideMarkdownPlugins::test_plugin_conversion
tests/test_transform_fields.py::TestGuideMarkdownPlugins::test_transform_guide_markdown_plugins
tests/test_utils.py::TestTimeUtils::test_caching
tests/test_utils.py::TestTimeUtils::test_event_class_normalization
tests/test_utils.py::TestTimeUtils::test_schema_multifields
tests/test_utils.py::TestTimeUtils::test_time_normalize
tests/test_version_locking.py::TestVersionLock::test_previous_entries_gte_current_min_stack
No tests found to execute!

detection_rules/config.py

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

detection_rules/custom_rules.py

…config cli

Mikaayenson · 2024-05-08T21:31:40Z

Alright! So after internal discussion here was what I implemented based on what we all agreed on.

init-config --> setup-config: Not just base file structure, but setup with initial file content so users can easily get to testing, and removed the config details since we're not defaulting to initialize files with content. Added a cli param to allow users to specify kibana version during setup. Also echoed doc links post setup.
SystemExit: Decided to keep this raised exit since the RULES_CONFIG = parse_rules_config() is initialized in so many places in the repo. Also because we dont have CLI context to try/catch.
Moved _config.yml validation to a dataclass (note: couldn't use mixins because of the multiple circular dependancies)
Added packages.yaml validation to the post init of the TestConfig
Updated doc references as needed.
Updated this PR summary to reflect the changes.

LMK if I missed anything!

detection_rules/config.py

brokensound77

Thanks for the changes - we're almost there; just a few remaining comments

detection_rules/config.py

detection_rules/custom_rules.py

Mikaayenson · 2024-05-09T13:37:49Z

I'd like to get @traut to put final 👀 on this but if it's a blocker to your PR, I can merge it in today @eric-forte-elastic.

eric-forte-elastic · 2024-05-09T13:47:50Z

I'd like to get @traut to put final 👀 on this but if it's a blocker to your PR, I can merge it in today @eric-forte-elastic.

I don't think its too much of a blocker to need to merge without @traut looking at it. I've just been merging in your changes to this branch into mine, so I can do the same thing, especially if we do not expect it to change much now. We will just need to merge this in before merging my PR.

detection_rules/config.py

detection_rules/custom_rules.py

brokensound77

a few notes, otherwise LGTM 🚀

detection_rules/config.py

traut

looks good!

Mikaayenson added 2 commits May 7, 2024 10:04

Improve dac custom init

ff96bea

Fix path naming

ae930e0

Mikaayenson added bug Something isn't working enhancement New feature or request detections-as-code labels May 7, 2024

Mikaayenson requested review from traut and eric-forte-elastic May 7, 2024 15:15

Mikaayenson self-assigned this May 7, 2024

Mikaayenson requested a review from brokensound77 as a code owner May 7, 2024 15:15

botelastic bot added the python Internal python for the repository label May 7, 2024

github-actions bot added the community label May 7, 2024

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/custom_rules.py Show resolved Hide resolved

Mikaayenson removed the community label May 7, 2024

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

patch for ci runs

a5776c5

eric-forte-elastic reviewed May 7, 2024

View reviewed changes

detection_rules/custom_rules.py Show resolved Hide resolved

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/main.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

add doc strings and rename test config name

d852520

eric-forte-elastic approved these changes May 7, 2024

View reviewed changes

expand how unit test are selected

a05894d

Mikaayenson commented May 7, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

raise unlink to CLI

536e4cd

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 7, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

Mikaayenson and others added 3 commits May 8, 2024 11:27

Update detection_rules/config.py

2070d9c

Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>

simplify package requirements

adf90e0

remove import

bdc17f0

brokensound77 reviewed May 8, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 8, 2024

View reviewed changes

detection_rules/custom_rules.py Show resolved Hide resolved

brokensound77 reviewed May 8, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 8, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 8, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

Mikaayenson added 3 commits May 8, 2024 16:16

add dataclass to validate rules config file and create default setup-…

8398554

…config cli

add kibana_version cli param

b4d849a

update doc string

f72744c

Mikaayenson commented May 8, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

Mikaayenson requested review from brokensound77 and eric-forte-elastic May 8, 2024 21:38

brokensound77 reviewed May 8, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

detection_rules/config.py Outdated Show resolved Hide resolved

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

rename delete cli option to overwrite, and small edits to exceptions

4c69c85

Mikaayenson mentioned this pull request May 9, 2024

update setup-config docs 3653 elastic/DaC-Reference#8

Merged

eric-forte-elastic approved these changes May 9, 2024

View reviewed changes

brokensound77 reviewed May 9, 2024

View reviewed changes

detection_rules/config.py Outdated Show resolved Hide resolved

brokensound77 reviewed May 9, 2024

View reviewed changes

detection_rules/custom_rules.py Outdated Show resolved Hide resolved

brokensound77 approved these changes May 9, 2024

View reviewed changes

traut reviewed May 10, 2024

View reviewed changes

detection_rules/config.py Show resolved Hide resolved

traut approved these changes May 10, 2024

View reviewed changes

type hint and small cleanup

0a66ee4

Mikaayenson merged commit ed34883 into DAC-feature May 10, 2024
14 checks passed

Mikaayenson deleted the 3621-frdac-raise-a-better-exception-for-missing-contents-in-config-files branch May 10, 2024 13:36

Mikaayenson mentioned this pull request May 17, 2024

[FR] Add --force flag to update-lock-versions #3693

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FR] Improve DAC custom folder init #3653

[FR] Improve DAC custom folder init #3653

Mikaayenson commented May 7, 2024 •

edited

Loading

brokensound77 commented May 7, 2024

Mikaayenson commented May 7, 2024

eric-forte-elastic left a comment

Mikaayenson commented May 8, 2024 •

edited

Loading

brokensound77 left a comment

Mikaayenson commented May 9, 2024

eric-forte-elastic commented May 9, 2024 •

edited

Loading

brokensound77 left a comment

traut left a comment

[FR] Improve DAC custom folder init #3653

[FR] Improve DAC custom folder init #3653

Conversation

Mikaayenson commented May 7, 2024 • edited Loading

Issues

Summary

Testing

brokensound77 commented May 7, 2024

Mikaayenson commented May 7, 2024

eric-forte-elastic left a comment

Choose a reason for hiding this comment

Mikaayenson commented May 8, 2024 • edited Loading

brokensound77 left a comment

Choose a reason for hiding this comment

Mikaayenson commented May 9, 2024

eric-forte-elastic commented May 9, 2024 • edited Loading

brokensound77 left a comment

Choose a reason for hiding this comment

traut left a comment

Choose a reason for hiding this comment

Mikaayenson commented May 7, 2024 •

edited

Loading

Mikaayenson commented May 8, 2024 •

edited

Loading

eric-forte-elastic commented May 9, 2024 •

edited

Loading