Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] kibana-upload KeyError: 'description' #239

Closed
SHolzhauer opened this issue Sep 2, 2020 · 2 comments
Closed

[Bug] kibana-upload KeyError: 'description' #239

SHolzhauer opened this issue Sep 2, 2020 · 2 comments
Labels
bug Something isn't working community

Comments

@SHolzhauer
Copy link
Contributor

Describe the bug
When tryin to use the kibana-upload command on a rule it throws an error:

(venv) user@machine:/path/to/detections$ python -m detection_rules kibana-upload --kibana-url https://kibana.tmp -u username rules/linux/rgn_iocuser_used.toml 

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

password: 
Loading rules from /path/to/detections/rules
Loaded 1 rules
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/path/to/detections/detection_rules/__main__.py", line 28, in <module>
    main()
  File "/path/to/detections/detection_rules/__main__.py", line 25, in main
    root(prog_name="detection_rules")
  File "/path/to/detections/venv/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File /path/to/detections/venv/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/path/to/detections/venv/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/path/to/detections/venv/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/path/to/detections/venv/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/path/to/detections/detection_rules/eswrap.py", line 264, in kibana_upload
    rules = RuleResource.bulk_create(api_payloads)
  File "/path/to/detections/kibana/resources.py", line 57, in bulk_create
    return [cls.from_dict(r) for r in responses]
  File "/path/to/detections/kibana/resources.py", line 57, in <listcomp>
    return [cls.from_dict(r) for r in responses]
  File "/path/to/detections/venv/lib/python3.6/site-packages/dataclasses_json/api.py", line 83, in from_dict
    return _decode_dataclass(cls, kvs, infer_missing)
  File "/path/to/detections/venv/lib/python3.6/site-packages/dataclasses_json/core.py", line 159, in _decode_dataclass
    field_value = kvs[field.name]
KeyError: 'description'

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'root directory'
  2. run python3 -m venv venv
  3. run source venv/bin/activate
  4. run pip install -r requirements.txt
Requirement already satisfied: jsl==0.2.4 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied: jsonschema==3.2.0 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 2))
Requirement already satisfied: pytoml in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 3))
Requirement already satisfied: toml==0.10.0 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 4))
Requirement already satisfied: requests==2.22.0 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 5))
Requirement already satisfied: Click==7.0 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 6))
Requirement already satisfied: PyYAML~=5.3 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 7))
Requirement already satisfied: dataclasses-json~=0.4.2 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 8))
Requirement already satisfied: eql~=0.9 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 9))
Requirement already satisfied: elasticsearch~=7.5.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 10))
Requirement already satisfied: pyflakes==2.2.0 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 13))
Requirement already satisfied: flake8==3.8.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 14))
Requirement already satisfied: pep8-naming==0.7.0 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 15))
Requirement already satisfied: pytest>=3.6 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 16))
Requirement already satisfied: importlib-metadata; python_version < "3.8" in ./venv/lib/python3.6/site-packages (from jsonschema==3.2.0->-r requirements.txt (line 2))
Requirement already satisfied: attrs>=17.4.0 in ./venv/lib/python3.6/site-packages (from jsonschema==3.2.0->-r requirements.txt (line 2))
Requirement already satisfied: six>=1.11.0 in ./venv/lib/python3.6/site-packages (from jsonschema==3.2.0->-r requirements.txt (line 2))
Requirement already satisfied: setuptools in ./venv/lib/python3.6/site-packages (from jsonschema==3.2.0->-r requirements.txt (line 2))
Requirement already satisfied: pyrsistent>=0.14.0 in ./venv/lib/python3.6/site-packages (from jsonschema==3.2.0->-r requirements.txt (line 2))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./venv/lib/python3.6/site-packages (from requests==2.22.0->-r requirements.txt (line 5))
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./venv/lib/python3.6/site-packages (from requests==2.22.0->-r requirements.txt (line 5))
Requirement already satisfied: idna<2.9,>=2.5 in ./venv/lib/python3.6/site-packages (from requests==2.22.0->-r requirements.txt (line 5))
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.6/site-packages (from requests==2.22.0->-r requirements.txt (line 5))
Requirement already satisfied: typing-inspect>=0.4.0 in ./venv/lib/python3.6/site-packages (from dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: marshmallow<4.0.0,>=3.3.0 in ./venv/lib/python3.6/site-packages (from dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: dataclasses; python_version == "3.6" in ./venv/lib/python3.6/site-packages (from dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: stringcase<2.0.0,==1.2.0 in ./venv/lib/python3.6/site-packages (from dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: marshmallow-enum<2.0.0,>=1.5.1 in ./venv/lib/python3.6/site-packages (from dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: lark-parser~=0.8.5 in ./venv/lib/python3.6/site-packages (from eql~=0.9->-r requirements.txt (line 9))
Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in ./venv/lib/python3.6/site-packages (from flake8==3.8.1->-r requirements.txt (line 14))
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in ./venv/lib/python3.6/site-packages (from flake8==3.8.1->-r requirements.txt (line 14))
Requirement already satisfied: flake8-polyfill<2,>=1.0.2 in ./venv/lib/python3.6/site-packages (from pep8-naming==0.7.0->-r requirements.txt (line 15))
Requirement already satisfied: py>=1.8.2 in ./venv/lib/python3.6/site-packages (from pytest>=3.6->-r requirements.txt (line 16))
Requirement already satisfied: iniconfig in ./venv/lib/python3.6/site-packages (from pytest>=3.6->-r requirements.txt (line 16))
Requirement already satisfied: packaging in ./venv/lib/python3.6/site-packages (from pytest>=3.6->-r requirements.txt (line 16))
Requirement already satisfied: pluggy<1.0,>=0.12 in ./venv/lib/python3.6/site-packages (from pytest>=3.6->-r requirements.txt (line 16))
Requirement already satisfied: more-itertools>=4.0.0 in ./venv/lib/python3.6/site-packages (from pytest>=3.6->-r requirements.txt (line 16))
Requirement already satisfied: zipp>=0.5 in ./venv/lib/python3.6/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema==3.2.0->-r requirements.txt (line 2))
Requirement already satisfied: mypy-extensions>=0.3.0 in ./venv/lib/python3.6/site-packages (from typing-inspect>=0.4.0->dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: typing-extensions>=3.7.4 in ./venv/lib/python3.6/site-packages (from typing-inspect>=0.4.0->dataclasses-json~=0.4.2->-r requirements.txt (line 8))
Requirement already satisfied: pyparsing>=2.0.2 in ./venv/lib/python3.6/site-packages (from packaging->pytest>=3.6->-r requirements.txt (line 16))

  1. create a TOML file ( i used the create command) with resulted in:
[metadata]
creation_date = "2020/08/29"
ecs_version = ["1.6.0"]
maturity = "development"
updated_date = "2020/08/29"

[rule]
author = ["S.Holzhauer"]
description = """
My fancy description
"""
language = "kuery"
license = "Elastic License"
name = "iocuser used"
risk_score = 1
rule_id = "b0eb4cf7-eaf4-43de-863b-34de522b518f"
severity = "low"
type = "query"

query = '''
user.name:iocuser
'''
  1. upload the rule python -m detection_rules kibana-upload --kibana-url https://kibana.tmp -u username rules/linux/rgn_iocuser_used.toml

Expected behavior
Detection rule to be uploaded to kibana and active

Screenshots
image

Desktop (please complete the following information):

  • OS: Linux
  • Version: Ubuntu 18.04

Additional context

@SHolzhauer SHolzhauer added the bug Something isn't working label Sep 2, 2020
@rw-access
Copy link
Contributor

I believe this bug was fixed with #121. Can you please update your branch and try again?

@SHolzhauer
Copy link
Contributor Author

Resolved! Tnx @rw-access

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working community
Projects
None yet
Development

No branches or pull requests

3 participants