Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix tls mapping in suricata module #19494

Merged
merged 4 commits into from
Jul 2, 2020
Merged

Fix tls mapping in suricata module #19494

merged 4 commits into from
Jul 2, 2020

Conversation

leehinman
Copy link
Contributor

What does this PR do?

Fixes tls mappings in suricata module. Specifically:

  • add suricata.eve.tls.ja3s.string field
  • add suricata.eve.tls.ja3s.hash field
  • add suricata.eve.tls.ja3.string field
  • add suricata.eve.tls.ja3.hash field
  • set default_field to false for suricata fields
  • map suricata.eve.tls.ja3.hash to tls.client.ja3
  • map suricata.eve.tls.ja3s.hash to tls.server.ja3s
  • perform suricata.eve.tls.* -> tls.* mappings for all event types

Why is it important?

  • If the tls.* mappings aren't filled in the event doesn't show up in
    the TLS tab in the SIEM
  • default_field to false to stay under 1000 fields

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=suricata mage -v pythonIntegTest

Related issues

@leehinman leehinman added bug Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM labels Jun 29, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 29, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 29, 2020 •
edited
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #19494 updated]

  • Start Time: 2020-07-02T13:37:32.333+0000

  • Duration: 27 min 53 sec

Test stats 🧪

Test Results
Failed 0
Passed 555
Skipped 128
Total 683

andrewkroh
andrewkroh approved these changes Jun 29, 2020
View reviewed changes
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I requested some changes but don't think it requires another review from me.

x-pack/filebeat/module/suricata/eve/config/eve.yml Outdated
- {from: suricata.eve.tls.chain, to: tls.server.certificate_chain}
- convert:
ignore_missing: true
ignore_failure: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ignore_failure: true
fail_on_error: false

x-pack/filebeat/module/suricata/eve/_meta/fields.yml Outdated
@@ -1,5 +1,6 @@
- name: eve
type: group
default_field: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid changing the behavior for the existing fields can you mark this on the two new field groups instead.

@leehinman leehinman force-pushed the leh_suricata_sdh branch 2 times, most recently from 6f27ebc to 00956eb Compare June 30, 2020 19:49
@leehinman
Fix tls mapping in suricata module
5ec28ba 
- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for suricata fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492
@leehinman
incorporate feedback
2eb76ac
@leehinman
Fix tests
ff025dc
@leehinman
Copy link
Contributor Author

run tests

@leehinman leehinman merged commit afffe2b into elastic:master Jul 2, 2020
@leehinman leehinman mentioned this pull request Jul 2, 2020
Merged
4 tasks
@leehinman leehinman added v7.7.2 and removed needs_backport PR is waiting to be backported to other branches. labels Jul 2, 2020
@leehinman leehinman mentioned this pull request Jul 2, 2020
Merged
4 tasks
@leehinman leehinman mentioned this pull request Jul 2, 2020
Merged
4 tasks
leehinman added a commit to leehinman/beats that referenced this pull request Jul 2, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
3fb028d 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit afffe2b)
leehinman added a commit that referenced this pull request Jul 2, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (#19494) (#19609)
0f0fe45 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes #19492

(cherry picked from commit afffe2b)
leehinman added a commit to leehinman/beats that referenced this pull request Jul 2, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
ef7ebb9 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit afffe2b)
v1v added a commit to v1v/beats that referenced this pull request Jul 3, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/windows-o…
0ba6f3b 
…ne-beats

* upstream/master: (35 commits)
  [ci] fix env variable name for xpack filebeats (elastic#19617)
  Cache error responses for cloudfoundry apps metadata (elastic#19181)
  ci: user fixed type of agent (elastic#19625)
  Input v2 cursor testing (elastic#19573)
  Update Jenkinsfile to not inspect removed vendor (elastic#19610)
  Fix ordering and duplicate configs on autodiscover (elastic#19317)
  Prepare input/file for changes in the registrar (elastic#19516)
  Cursor input and manager implementation (elastic#19571)
  [Filebeat] Fix tls mapping in suricata module (elastic#19494)
  [Ingest Manager] Make Agent beta and Constraints experimental (elastic#19586)
  Accept prefix as metric_types for stackdriver metricset in GCP (elastic#19345)
  Implement memlog store operations (elastic#19533)
  introduce journalbeat/pkg in order to provide reusable shared code (elastic#19581)
  Add descriptions to HAProxy fields in Metricbeat (elastic#19561)
  ci: apm-server-update trigered only on upstream, comments, and manual triggered (elastic#19590)
  ci: enable upstream triggering on the packaging job (elastic#19589)
  ci: some jjbb improvements (elastic#19588)
  [MetricBeat] set tags correctly if the dimension value is ARN (elastic#19433)
  [Filebeat] Add default_fields: false to fields.yml in aws module (elastic#19568)
  Add publisher implementation for stateful inputs (elastic#19530)
  ...
leehinman added a commit to leehinman/beats that referenced this pull request Jul 6, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
bfa2bb3 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit afffe2b)
leehinman added a commit that referenced this pull request Jul 6, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (#19494) (#19608)
0f2144a 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes #19492

(cherry picked from commit afffe2b)
leehinman added a commit that referenced this pull request Jul 6, 2020
@leehinman
[Filebeat] Fix tls mapping in suricata module (#19494) (#19607)
4f4baef 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes #19492

(cherry picked from commit afffe2b)
@leehinman leehinman deleted the leh_suricata_sdh branch October 5, 2020 19:21
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
@leehinman @melchiormoulin
[Filebeat] Fix tls mapping in suricata module (elastic#19494)
89eea04 
* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494) (elasti…
a4fb441 
…c#19607)

* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit 362016d)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@leehinman
[Filebeat] Fix tls mapping in suricata module (elastic#19494) (elasti…
da3007a 
…c#19608)

* Fix tls mapping in suricata module

- add suricata.eve.tls.ja3s.string field
- add suricata.eve.tls.ja3s.hash field
- add suricata.eve.tls.ja3.string field
- add suricata.eve.tls.ja3.hash field
- set default_field to false for ja3 & ja3s fields
- map suricata.eve.tls.ja3.hash to tls.client.ja3
- map suricata.eve.tls.ja3s.hash to tls.server.ja3s
- perform suricata.eve.tls.* -> tls.* mappings for all event types

Closes elastic#19492

(cherry picked from commit 362016d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees
No one assigned
Labels
bug Filebeat Filebeat v7.7.2 v7.8.1 v7.9.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Filebeat] suricata fileset doesn't capture tls fields for alerts
3 participants
@leehinman @elasticmachine @andrewkroh