[Bug] Kibana upload getting unauthorized error

[shravaka]

Description:
It is impossible to upload rules to a internally or locally hosted instance by using the Python script.
The cause seems to be one of the parameters in line 118 of the /detection-rules/kibana/connector.py file:
payload = {'params': payload, 'currentURL': '', 'providerType': 'basic', 'providerName': 'cloud-basic'}

Should the 'ProviderName' be hardcoded here?

Steps to reproduce the behavior:

1. Execute the command:
   python -m detection_rules kibana --kibana-url xxx -ku username -kp password upload-rule example.toml
   where xxx is internally hosted instance of kibana

Expected behavior:
Rule successfully uploaded to Kibana

Actual behavior:
Python returns an error:
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url:xxx

• OS: Ubuntu
• Version: 20.04
• OS: Centos
• Version 7.8

[rw-access]

@shravaka based off the 401 error message, it seems like this is related to credentials, or the permissions of the account you'e using. Can you share more of the stack trace from your terminal output? This would help us pinpoint the bug better.

And please don't forget to remove any personal information, like your credentials from the output.

[shravaka]

Yeah i thought so too initially and wasted much time on double checking permissions and password, but in the end managed to push the files after changing 'providerName': 'cloud-basic' to simply 'providerName': 'basic'

I'll share stack shortly

[shravaka]

`$ python3 -m detection_rules -D kibana --kibana-url [kibanaurl] -ku [user] -kp [password] upload-rule ./rules/*.toml

DEBUG MODE ENABLED
{"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}
Traceback (most recent call last):
  File "/detection-rules/kibana/connector.py", line 114, in login
    self.post(path, data=payload, error=True, verbose=False)
  File "/detection-rules/kibana/connector.py", line 98, in post
    return self.request('POST', uri, params=params, data=data, error=error, **kwargs)
  File "/detection-rules/kibana/connector.py", line 77, in request
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: [kibanaurl]/internal/security/login

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/detection-rules/detection_rules/__main__.py", line 28, in <module>
    main()
  File "/detection-rules/detection_rules/__main__.py", line 25, in main
    root(prog_name="detection_rules")
  File "[homedir]/.local/lib/python3.8/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "[homedir]/.local/lib/python3.8/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "[homedir]/.local/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[homedir]/.local/lib/python3.8/site-packages/click/core.py", line 1134, in invoke
    Command.invoke(self, ctx)
  File "[homedir]/.local/lib/python3.8/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "[homedir]/.local/lib/python3.8/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "[homedir]/.local/lib/python3.8/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/detection-rules/detection_rules/kbwrap.py", line 43, in kibana_group
    ctx.obj['kibana'] = get_kibana_client(**kibana_kwargs)
  File "/detection-rules/detection_rules/kbwrap.py", line 26, in get_kibana_client
    kibana.login(kibana_user, kibana_password)
  File "/detection-rules/kibana/connector.py", line 119, in login
    self.post(path, data=payload, error=True)
  File "/detection-rules/kibana/connector.py", line 98, in post
    return self.request('POST', uri, params=params, data=data, error=error, **kwargs)
  File "/detection-rules/kibana/connector.py", line 77, in request
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: [kibanaurl]/internal/security/login
`

[operatorequals]

Had a very similar case and debugged it on Slack:
https://elasticstack.slack.com/archives/C016E72DWDS/p1606289731228400

My version is 7.10

[shravaka]

Forgot to mention that version I'm working with is 7.10 as well