Update to newest rules in Kibana

[swiftbird07]

Hello,

I just want to update all the rules in Kibana to the rules I see here.
What would be the easiest way to do that?

I tried using the CLI python method but it fails to export the rules to the .ndjson file

Example:
CLI Error: Unknown rules for rule IDs: rules/macos/credential_access_access_to_browser_credentials_procargs.toml

using the -d option results in a strange error:

root@fort-noggs-s2ubntLT:/etc/elk_rules_update/detection-rules# python3 -m detection_rules export-rules -d rules/macos/ -o expooort.ndjson -s -r

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/etc/elk_rules_update/detection-rules/detection_rules/__main__.py", line 34, in <module>
    main()
  File "/etc/elk_rules_update/detection-rules/detection_rules/__main__.py", line 31, in main
    root(prog_name="detection_rules")
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/etc/elk_rules_update/detection-rules/detection_rules/main.py", line 246, in export_rules
    rule.contents['rule_id'] = str(uuid4())
TypeError: 'TOMLRuleContents' object does not support item assignment

As you can see I already tried the -s -r options.

I also tried to upload the rules directly to Kibana but that resulted in another error:

python3 -m detection_rules kibana -ku elastic -kp verygoodpw --kibana-url http://192.168.178.83:5601 upload-rule rule/macos/*.toml
[...]
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http://192.168.178.83:5601/internal/security/login

Why is this so hard? Am I overseeing something?

Btw. I have the standalone version (non cloud) if this is important.

[brokensound77]

Hello @maof97, thanks for opening the issue and sorry for the frustrations. There are a few things going on here.

The current release process merges all these rules to main, which will then be merged into Kibana by the next stack release (7.13 in this case). There is an ongoing issue (#362) for better incorporating a sync model using git for users who want to integrate rules into their workflow. Additionally, we will begin releasing rules separate from Kibana releases very soon, which would allow them to be incorporated much quicker.

The error for export-rules looks like it was introduced in a commit this week, and so I will take a look at that.

The error for upload-rule is a known error (#634) resulting from logging into a non-cloud instance. I will get that addressed too.

[swiftbird07]

Thank you for your friendly and fast support! I will look forward to the mentioned features/changes.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.