Filebeat Cisco module doesn't correctly parse 305011, 302015, 302013, or 722051 message types #21658
Comments
botelastic
bot
added
the
needs_team
botelastic
bot
removed
the
needs_team
rjthielen
changed the title
|
I believe that I have a suggested fix for the issue. When I tried to fix the patterns before, I didn't realize that I needed to specify |
|
I am still having an issue even after using this latest yml file and updated pipeline as stated above. Blow is the warning message keeps recording in logstash [2020-10-30T00:51:29,202][WARN ][logstash.outputs.elasticsearch][asa][d416fabea07ba3f13eab657ed7767e90bb3b5962d9c68b1d2bd09dbc9d06a511] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"cisco-asa-2020.10.30", :routing=>nil, :_type=>"_doc", :pipeline=>"filebeat-7.9.2-cisco-asa-asa-ftd-pipeline"}, #LogStash::Event:0x3674e4ee], :response=>{"index"=>{"_index"=>"cisco-asa-2020.10.30", "_type"=>"_doc", "_id"=>"Iiz8dnUBrVYcZmTIrgtm", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.port] of type [long] in document with id 'Iiz8dnUBrVYcZmTIrgtm'. Preview of field's value: '50185(LOCAL\abc10686)'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"For input string: "50185(LOCAL\abc10686)""}}}}} |
Fixes elastic#21658 For messages 716002: - Changed to GROK; allows for better parsing of event.reason - Added field for cisco.webvpn.group_name - Added field for event.reason per cisco docs for why session was terminated - Added field for cisco.termination_user for the AAA username terminating the connection For messages 722051: - Add angle brackets to dissect to properly dissect the message, per cisco docs - Added field for cisco.webvpn.group.name For messages 305011: - Change to GROK; allows for variance in message format with identity firewall For messages 302020: - Added GROK pattern to allows for variance in message format with identity firewall For messages 302014/302016/302021: - Added patterns and modified order of patterns of GROK to better match teardown messages - Note that order of processing is important as the most specific messages are matched first, falling through to the appropriate match. - Added temp fields for teardown initiator and user; defined in cisco docs but currently no real place to put them, but could be in future. - Added icmp_type and icmp_code parsing for 302021 messages - Changed duration matching from TIME to NOTSPACE, as long-lived connections (over 24 hours) don't match TIME. And: - Added descriptions to many fields to make them easier to find in 7.9+ Kibana Ingest Node Pipeline editor. - Changed source.bytes field type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long. - Changed destination.bytes type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long.
Fixes #21658 For messages 716002: - Changed to GROK; allows for better parsing of event.reason - Added field for cisco.webvpn.group_name - Added field for event.reason per cisco docs for why session was terminated - Added field for cisco.termination_user for the AAA username terminating the connection For messages 722051: - Add angle brackets to dissect to properly dissect the message, per cisco docs - Added field for cisco.webvpn.group.name For messages 305011: - Change to GROK; allows for variance in message format with identity firewall For messages 302020: - Added GROK pattern to allows for variance in message format with identity firewall For messages 302014/302016/302021: - Added patterns and modified order of patterns of GROK to better match teardown messages - Note that order of processing is important as the most specific messages are matched first, falling through to the appropriate match. - Added temp fields for teardown initiator and user; defined in cisco docs but currently no real place to put them, but could be in future. - Added icmp_type and icmp_code parsing for 302021 messages - Changed duration matching from TIME to NOTSPACE, as long-lived connections (over 24 hours) don't match TIME. And: - Added descriptions to many fields to make them easier to find in 7.9+ Kibana Ingest Node Pipeline editor. - Changed source.bytes field type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long. - Changed destination.bytes type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long.
Fixes elastic#21658 For messages 716002: - Changed to GROK; allows for better parsing of event.reason - Added field for cisco.webvpn.group_name - Added field for event.reason per cisco docs for why session was terminated - Added field for cisco.termination_user for the AAA username terminating the connection For messages 722051: - Add angle brackets to dissect to properly dissect the message, per cisco docs - Added field for cisco.webvpn.group.name For messages 305011: - Change to GROK; allows for variance in message format with identity firewall For messages 302020: - Added GROK pattern to allows for variance in message format with identity firewall For messages 302014/302016/302021: - Added patterns and modified order of patterns of GROK to better match teardown messages - Note that order of processing is important as the most specific messages are matched first, falling through to the appropriate match. - Added temp fields for teardown initiator and user; defined in cisco docs but currently no real place to put them, but could be in future. - Added icmp_type and icmp_code parsing for 302021 messages - Changed duration matching from TIME to NOTSPACE, as long-lived connections (over 24 hours) don't match TIME. And: - Added descriptions to many fields to make them easier to find in 7.9+ Kibana Ingest Node Pipeline editor. - Changed source.bytes field type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long. - Changed destination.bytes type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long. (cherry picked from commit b5e43fc)
Fixes #21658 For messages 716002: - Changed to GROK; allows for better parsing of event.reason - Added field for cisco.webvpn.group_name - Added field for event.reason per cisco docs for why session was terminated - Added field for cisco.termination_user for the AAA username terminating the connection For messages 722051: - Add angle brackets to dissect to properly dissect the message, per cisco docs - Added field for cisco.webvpn.group.name For messages 305011: - Change to GROK; allows for variance in message format with identity firewall For messages 302020: - Added GROK pattern to allows for variance in message format with identity firewall For messages 302014/302016/302021: - Added patterns and modified order of patterns of GROK to better match teardown messages - Note that order of processing is important as the most specific messages are matched first, falling through to the appropriate match. - Added temp fields for teardown initiator and user; defined in cisco docs but currently no real place to put them, but could be in future. - Added icmp_type and icmp_code parsing for 302021 messages - Changed duration matching from TIME to NOTSPACE, as long-lived connections (over 24 hours) don't match TIME. And: - Added descriptions to many fields to make them easier to find in 7.9+ Kibana Ingest Node Pipeline editor. - Changed source.bytes field type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long. - Changed destination.bytes type from integer to long, since long-lived flows can surpass the capacity of an integer; ECS reference field is defined as long. (cherry picked from commit b5e43fc) Co-authored-by: hitchatwork <56573831+hitchatwork@users.noreply.github.com>
Version: 7.9.2
Platform: Debian 10.5
cf. https://discuss.elastic.co/t/filebeat-cisco-module-cant-parse-305011-302015-302013-or-722015-message-types/250882
Related issue: https://github.com/elastic/beats/pull/17964
Summary: dissect patterns in asa-ftd-pipeline.yml do not parse Cisco message numbers 305011, 302015, or 302013 when a user identity is present.
Recommended Solution: replace dissect patterns with grok patterns if multiple variations are required.
Note that the Logstash Cisco grok patterns for the 305011 and 302... messages correctly parse them.
The Cisco documentation makes it clear that the user name may be present in the log messages for all three of these message types. https://www.cisco.com/c/en/us/td/docs/security/asa/syslog/b_syslog/syslog-messages-302003-to-342008.html
Sample messages:
%ASA-6-305011:
Built dynamic UDP translation from any:10.10.142.244/62776(LOCAL\username) to USER-VLAN1120:10.11.121.66/62776 Built dynamic TCP translation from any:10.10.128.177/50350 to VLAN1120:10.11.121.118/50350%ASA-6-302013:
Built inbound TCP connection 2568541635 for VLAN1120:10.10.192.226/52232 (10.11.121.28/52232)(LOCAL\username1) to VLAN1120:172.217.4.46/443 (172.217.4.46/443) (username1)Built inbound TCP connection 2568541636 for VLAN1120:10.10.142.144/54095 (10.11.121.70/54095) to VLAN1120:172.217.4.46/443 (172.217.4.46/443) (username2)Built outbound TCP connection 2568541648 for VLAN1120:10.10.138.94/54582 (10.10.138.94/54582) to identity:172.217.4.46/443 (172.217.4.46/443)%ASA-6-302015:
Built inbound UDP connection 2568541626 for VLAN1120:10.10.142.244/62776 (10.11.121.66/62776)(LOCAL\username) to VLAN1120:172.217.4.46/443 (172.217.4.46/443) (username)Furthermore, message 722051 doesn't parse correctly when there are angle brackets separating field values in the text, e.g.
Group <group-policy> User <user1> IP <10.10.10.10> IPv4 Address <10.11.11.11> IPv6 address <::> assigned to sessionThe text was updated successfully, but these errors were encountered: