[Filebeat] Cisco FTD issues parsing Security Event messages #16889
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/siem (Team:SIEM) |
andrewstucki
commented
Mar 6, 2020
| - kv: | ||
| if: '["430001", "430002", "430003", "430004", "430005", ""].contains(ctx._temp_.cisco.message_id)' | ||
| field: "message" | ||
| field_split: ",(?=\\s[A-za-z1-9\\s]+:)" |
There was a problem hiding this comment.
This was causing some fields to get dropped when field values contained a , character in them. The regex portion is a lookahead to make sure we have a space followed by a valid field name (all letters, numbers, and spaces) with a colon at the end.
andrewstucki
commented
Mar 6, 2020
| patterns: | ||
| - "(?:%{SYSLOG_HEADER})?\\s*%{GREEDYDATA:log.original}" | ||
| pattern_definitions: | ||
| SYSLOG_HEADER: "(?:%{SYSLOGFACILITY}\\s*)?(?:%{FTD_DATE:_temp_.raw_date}:?\\s+)?(?:%{PROCESS_HOST}|%{HOST_PROCESS})(?:{DATA})?%{SYSLOG_END}?" |
There was a problem hiding this comment.
The lack of an optional SYSLOG_END was causing timestamps to get truncated in the test log. Furthermore, some logs have process prior to hostname, so added an alternative swapping of the hostname and process.
andrewstucki
commented
Mar 6, 2020
| - grok: | ||
| field: log.original | ||
| patterns: | ||
| - "%{FTD_PREFIX}-(?:%{FTD_SUFFIX:_temp_.cisco.suffix}-)?%{NONNEGINT:event.severity:int}-%{POSINT:_temp_.cisco.message_id}?:?\\s*%{GREEDYDATA:message}" |
There was a problem hiding this comment.
The use of POSINT was causing cases of -0- severity to get misinterpreted.
adriansr
approved these changes
Mar 12, 2020
Merged
4 tasks
Merged
4 tasks
andrewstucki
pushed a commit
that referenced
this pull request
Mar 19, 2020
…ty Event messages (#16981) * [Filebeat] Cisco FTD issues parsing Security Event messages (#16889) * Fix grok and kv split bugs * Fix optional whitespace for field name separator (cherry picked from commit 912eac4) * [Filebeat] Add changelog entry for Cisco fixes (#17124) * Add changelog entry for Cisco fixes * move new entry to the end of the changelog section * Remove stray changelog entries from cherry-pick
andrewstucki
pushed a commit
that referenced
this pull request
Apr 4, 2020
…ty Event messages (#16982) * [Filebeat] Cisco FTD issues parsing Security Event messages (#16889) * Fix grok and kv split bugs * Fix optional whitespace for field name separator (cherry picked from commit 912eac4) * [Filebeat] Add changelog entry for Cisco fixes (#17124) * Add changelog entry for Cisco fixes * move new entry to the end of the changelog section * Remove stray changelog entries from cherry-pick
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
You'll want to take a look at this pr with
?w=1since there was a lot of formatting changes for the pipeline.What does this PR do?
Fixes a few bugs with the shared cisco parsing pipeline that was causing some fields to be dropped/misinterpreted
Checklist
Related issues