Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bitdefender: Add jsonRPC format as recommended default. #10381

Merged
merged 11 commits into from
Jul 26, 2024

Conversation

kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Jul 4, 2024

Proposed commit message

Add support for `jsonRPC` format.

After input limitation in processing `jsonRPC` is removed, this PR 
adds support to process `jsonRPC` formatted events and also suggests 
making this as default format as recommended by BitDefender.

   - Update docs to indicate `jsonRPC` is recommended default.

   - Increment major version to support this breaking change 
     making jsonRPC as default recommended.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • Add system tests to jsonrpc.
  • Test against live data setting up BitDefender agent.
  • Update README to remove qradar as default.

How to test this PR locally

$ eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v --data-streams=push_notifications

--- Test results for package: bitdefender - START ---
╭─────────────┬────────────────────┬───────────┬────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM        │ TEST TYPE │ TEST NAME                                                      │ RESULT │ TIME ELAPSED │
├─────────────┼────────────────────┼───────────┼────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ bitdefender │ push_notifications │ pipeline  │ (ingest pipeline warnings test-push-notification-jsonrpc.json) │ PASS   │ 505.980333ms │
│ bitdefender │ push_notifications │ pipeline  │ (ingest pipeline warnings test-push-notification.json)         │ PASS   │ 507.769209ms │
│ bitdefender │ push_notifications │ pipeline  │ test-push-notification-jsonrpc.json                            │ PASS   │ 341.265167ms │
│ bitdefender │ push_notifications │ pipeline  │ test-push-notification.json                                    │ PASS   │ 265.983791ms │
╰─────────────┴────────────────────┴───────────┴────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: bitdefender - END   ---
Done

$ eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v --data-streams=push_notifications

--- Test results for package: bitdefender - START ---
╭─────────────┬────────────────────┬───────────┬─────────────────────────────────────────────┬────────┬───────────────╮
│ PACKAGE     │ DATA STREAM        │ TEST TYPE │ TEST NAME                                   │ RESULT │  TIME ELAPSED │
├─────────────┼────────────────────┼───────────┼─────────────────────────────────────────────┼────────┼───────────────┤
│ bitdefender │ push_notifications │ system    │ bitdefender-push-notification-jsonrpc-http  │ PASS   │ 41.453836833s │
│ bitdefender │ push_notifications │ system    │ bitdefender-push-notification-jsonrpc-https │ PASS   │ 40.982260833s │
│ bitdefender │ push_notifications │ system    │ bitdefender-push-notification-qradar-http   │ PASS   │ 40.880906709s │
╰─────────────┴────────────────────┴───────────┴─────────────────────────────────────────────┴────────┴───────────────╯
--- Test results for package: bitdefender - END   ---
Done

Related issues

Screenshots

Screenshot 2024-07-23 at 10 55 53 PM

@andrewkroh andrewkroh added Integration:bitdefender BitDefender Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations] labels Jul 19, 2024
@kcreddy kcreddy changed the title bitdefender: Process jsonRPC messages using CEL bitdefender: Add jsonRPC format as recommended default. Jul 23, 2024
@kcreddy kcreddy added the enhancement New feature or request label Jul 23, 2024
@kcreddy kcreddy marked this pull request as ready for review July 23, 2024 17:53
@kcreddy kcreddy requested a review from a team as a code owner July 23, 2024 17:53
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

Comment on lines 44 to 53
{{else}}
program: |
has(obj.jsonrpc) ?
dyn({
"error": {
"message": "Unable to process message. Received jsonrpc formatted message, but setting `Enable jsonRPC Format` is disabled."
}
})
:
obj
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this part; the CEL processing is more expensive than the direct publication. Do you think that the error message here carries its weight beyond what would be seen without? A possible alternative seems like it should be possible in the ingest pipeline; test there for the presence of the json.jsonrpc field and add this same message then. The difference would be that the agent doesn't know about it. This difference may be significant when the status updates are added to the http_endpoint input.

Copy link
Contributor Author

@kcreddy kcreddy Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up removing the user option to enable/disable the jsonrpc format. The input now parses all 3 formats jsonrpc, qradar or splunk without any fail processors in ingest pipeline. Change in 5867fdf. Let me know if this sounds good.

- name: enable_jsonrpc_format
type: bool
title: Enable jsonRPC Format.
description: Enable this flag if events are in BitDefender's jsonRPC format. For details, see [documentation](https://www.bitdefender.com/business/support/en/77209-135325-push-event-json-rpc-messages.html).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text here says "enable this … if", but it is already on by default. Suggest "Enable processing events in BitDefender's jsonRPC format."?

Do we want this option on by default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will change the description. Initially I added it as default: false and its description accordingly.
But after talking to Bitdefender folks, they recommend using this format as default. So, I changed the default: true and incremented major version to indicate this is a breaking change. I forgot to update the description accordingly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this option all together
#10381 (comment)

@elasticmachine
Copy link

elasticmachine commented Jul 25, 2024

🚀 Benchmarks report

Package bitdefender 👍(2) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
push_configuration 29411.76 23809.52 -5602.24 (-19.05%) 💔

To see the full report comment with /test benchmark fullreport

Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after nits.

packages/bitdefender/_dev/build/docs/README.md Outdated Show resolved Hide resolved
packages/bitdefender/docs/README.md Outdated Show resolved Hide resolved
@elasticmachine
Copy link

💚 Build Succeeded

History

@kcreddy kcreddy merged commit 66f1e59 into elastic:main Jul 26, 2024
5 checks passed
@elasticmachine
Copy link

Package bitdefender - 2.0.0 containing this change is available at https://epr.elastic.co/search?package=bitdefender

jvalente-salemstate pushed a commit to jvalente-salemstate/integrations that referenced this pull request Aug 21, 2024
Add support for `jsonRPC` format.

After input limitation in processing `jsonRPC` is removed, this PR 
adds support to process `jsonRPC` formatted events and also suggests 
making this as default format as recommended by BitDefender.

   - Update docs to indicate `jsonRPC` is recommended default.

   - Increment major version to support this breaking change 
     making jsonRPC as default recommended.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:bitdefender BitDefender Team:Security-Service Integrations Security Service Integrations Team [elastic/security-service-integrations]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bitdefender: Support jsonRPC format messages in Bitdefender
4 participants