Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Winlogbeat Could not index event - "source.ip: 'LOCAL'" Not an IP String Literal #19627

Closed
MakoWish opened this issue Jul 2, 2020 · 12 comments · Fixed by #21325
Closed

Winlogbeat Could not index event - "source.ip: 'LOCAL'" Not an IP String Literal #19627

MakoWish opened this issue Jul 2, 2020 · 12 comments · Fixed by #21325
Labels
bug Team:Integrations Label for the Integrations team

Comments

@MakoWish
Copy link
Contributor

MakoWish commented Jul 2, 2020

I am seeing quite a lot of errors in Logstash for Winlogbeat events failing to index. The error indicates events are coming in with "source.ip: LOCAL" which is not a valid IP address. If for some reason the literal string for the IP address is being read as "LOCAL", the agent should convert this to "127.0.0.1" to prevent index failures.

Jul 02 07:43:04 Logstash1 logstash[7790]: [2020-07-02T07:43:04,974][WARN ][logstash.outputs.elasticsearch][main][1b0d38a63ac70b958df647ae2a47badf4ac8161e6df5e7eb331817d2b52dfa28] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.8.0", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x5a7b7089>], :response=>{"index"=>{"_index"=>"winlogbeat-7.8.0-2020.07.02-000015", "_type"=>"_doc", "_id"=>"-nL7D3MB9q2MOx9CKDBo", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id '-nL7D3MB9q2MOx9CKDBo'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}}
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 2, 2020
@jsoriano jsoriano added [zube]: Inbox bug Team:Integrations Label for the Integrations team Team:SIEM and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 3, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@zube zube bot removed the Winlogbeat label Jul 3, 2020
@andrewkroh
Copy link
Member

Could you please provide a copy of the event being indexed as well as the configuration in use with Winlogbeat. We need to be able to identify what module and event ID was being processed when this occurred. Typically we are validating IPs fields before copying them into the source.ip and destination.ip field, but perhaps a module is missing this extra validation in some place.

@MakoWish
Copy link
Contributor Author

Unfortunately, I cannot provide a copy of the events being indexed, as they are failing to index, and there is no indication of what device(s) the events are coming from. Based on the number of times this error is shown, I would assume the events are coming from all devices. I can, however, give you as many examples of the errors from Logstash as you want. There are hundreds of these every day.

[2020-07-13T05:03:27,799][WARN ][logstash.outputs.elasticsearch][main][1b0d38a63ac70b958df647ae2a47badf4ac8161e6df5e7eb331817d2b52dfa28] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.8.0", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x609729cb>], :response=>{"index"=>{"_index"=>"winlogbeat-7.8.0-2020.07.13-000031", "_type"=>"_doc", "_id"=>"79kOSHMB3sqroDPy-UOo", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id '79kOSHMB3sqroDPy-UOo'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}}
[2020-07-13T06:04:17,731][WARN ][logstash.outputs.elasticsearch][main][1b0d38a63ac70b958df647ae2a47badf4ac8161e6df5e7eb331817d2b52dfa28] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.8.0", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x641e4478>], :response=>{"index"=>{"_index"=>"winlogbeat-7.8.0-2020.07.13-000031", "_type"=>"_doc", "_id"=>"MO5GSHMByJYjKOHWq3Uu", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id 'MO5GSHMByJYjKOHWq3Uu'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}}
[2020-07-13T06:29:16,891][WARN ][logstash.outputs.elasticsearch][main][1b0d38a63ac70b958df647ae2a47badf4ac8161e6df5e7eb331817d2b52dfa28] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.8.0", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x2a2a3982>], :response=>{"index"=>{"_index"=>"winlogbeat-7.8.0-2020.07.13-000031", "_type"=>"_doc", "_id"=>"LfZdSHMByJYjKOHWi8E_", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id 'LfZdSHMByJYjKOHWi8E_'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}}
[2020-07-13T06:01:23,766][WARN ][logstash.outputs.elasticsearch][main][1b0d38a63ac70b958df647ae2a47badf4ac8161e6df5e7eb331817d2b52dfa28] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.8.0", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x43c8bed3>], :response=>{"index"=>{"_index"=>"winlogbeat-7.8.0-2020.07.13-000031", "_type"=>"_doc", "_id"=>"Ze1ESHMByJYjKOHWA4qm", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id 'Ze1ESHMByJYjKOHWA4qm'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}}

@MakoWish
Copy link
Contributor Author

Still looking for a fix on this. Any update? We are now running Winlogbeat 7.9.1 and the issue is still present.

Sep 23 07:50:13 Logstash1 logstash[10549]: [2020-09-23T07:50:13,796][WARN ][logstash.outputs.elasticsearch][main][b30fd535996a1206b29059a39c76d22d08adf45d978f1d84f5f2c2dcef72ecc5] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.9.1", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x3b3347bd>], :response=>{"index"=>{"_index"=>"winlogbeat-7.9.1-2020.09.22-000005", "_type"=>"_doc", "_id"=>"ruZxu3QBbTYEwL8ahSRb", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id 'ruZxu3QBbTYEwL8ahSRb'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}}

@andrewkroh
Copy link
Member

Can you try setting up a dead letter queue to capture the event that's failing to index. https://www.elastic.co/guide/en/logstash/current/dead-letter-queues.html

@MakoWish
Copy link
Contributor Author

@andrewkroh, Please see below. Redacted computer, user, and domain names, but these messages are from three different devices; all running 7.9.1 Winlogbeat.

1c  �nÿÿÿÿp&Ë‘   �2020-09-24T15:23:31.211Z  ¡Ÿqjava.util.HashMap¿dDATAŸx�org.logstash.ConvertedMap¿dhostŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringw<redacted>.<redacted>.COMÿhhostnameŸtorg.jruby.RubyStringh<redacted>ÿÿÿfsourceŸx�org.logstash.ConvertedMap¿cgeoŸx�org.logstash.ConvertedMap¿ÿÿbipŸtorg.jruby.RubyStringeLOCALÿfdomainŸtorg.jruby.RubyStringgUnknownÿÿÿcecsŸx�org.logstash.ConvertedMap¿gversionŸtorg.jruby.RubyStringe1.5.0ÿÿÿduserŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringl<redacted>ÿfdomainŸtorg.jruby.RubyStringe<redacted>ÿÿÿclogŸx�org.logstash.ConvertedMap¿elevelŸtorg.jruby.RubyStringkinformationÿÿÿgmessageŸtorg.jruby.RubyStringy�žA session was disconnected from a Window Station.

Subject:
	Account Name:		<redacted>
	Account Domain:		<redacted>
	Logon ID:		0x259a71

Session:
	Session Name:		Console

Additional Information:
	Client Name:		Unknown
	Client Address:		LOCAL


This event is generated when a user disconnects from an existing Terminal Services session, or when a user switches away from an existing desktop using Fast User Switching.ÿfwinlogŸx�org.logstash.ConvertedMap¿hevent_id�      �«gchannelŸtorg.jruby.RubyStringhSecurityÿjevent_dataŸx�org.logstash.ConvertedMap¿gLogonIDŸtorg.jruby.RubyStringh0x259a71ÿmClientAddressŸtorg.jruby.RubyStringeLOCALÿkSessionNameŸtorg.jruby.RubyStringgConsoleÿjClientNameŸtorg.jruby.RubyStringgUnknownÿmAccountDomainŸtorg.jruby.RubyStringe<redacted>ÿkAccountNameŸtorg.jruby.RubyStringl<redacted>ÿÿÿmprovider_guidŸtorg.jruby.RubyStringx&{54849625-5478-4994-A5BA-3E3B0328C30D}ÿmcomputer_nameŸtorg.jruby.RubyStringw<redacted>.<redacted>.comÿfopcodeŸtorg.jruby.RubyStringdInfoÿhkeywordsŸx�org.logstash.ConvertedListŸŸtorg.jruby.RubyStringmAudit SuccessÿÿÿdtaskŸtorg.jruby.RubyStringx�Other Logon/Logoff Eventsÿirecord_id�      �ÂcapiŸtorg.jruby.RubyStringkwineventlogÿmprovider_nameŸtorg.jruby.RubyStringx#Microsoft-Windows-Security-AuditingÿgprocessŸx�org.logstash.ConvertedMap¿fthreadŸx�org.logstash.ConvertedMap¿bid�      �\ÿÿcpid�      �0ÿÿelogonŸx�org.logstash.ConvertedMap¿bidŸtorg.jruby.RubyStringh0x259a71ÿÿÿÿÿdtagsŸx�org.logstash.ConvertedListŸŸtorg.jruby.RubyStringx�beats_input_codec_plain_appliedÿÿÿh@versiona1grelatedŸx�org.logstash.ConvertedMap¿duserŸtorg.jruby.RubyStringl<redacted>ÿÿÿeagentŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringh<redacted>ÿgversionŸtorg.jruby.RubyStringe7.9.1ÿlephemeral_idŸtorg.jruby.RubyStringx$af57a7de-e59a-445f-9320-1e15053adb37ÿdtypeŸtorg.jruby.RubyStringjwinlogbeatÿbidŸtorg.jruby.RubyStringx$7c23fe19-c4f0-423d-94fd-5952dee5b471ÿhhostnameŸtorg.jruby.RubyStringh<redacted>ÿÿÿj@timestampŸvorg.logstash.Timestampx�2020-09-24T15:23:27.916ZÿeeventŸx�org.logstash.ConvertedMap¿dkindŸtorg.jruby.RubyStringeeventÿhproviderŸtorg.jruby.RubyStringx#Microsoft-Windows-Security-AuditingÿfmoduleŸtorg.jruby.RubyStringhsecurityÿfactionŸtorg.jruby.RubyStringtsession-disconnectedÿhcategoryŸtorg.jruby.RubyStringnauthenticationÿgcreatedŸtorg.jruby.RubyStringx�2020-09-24T15:23:29.806ZÿdtypeŸtorg.jruby.RubyStringcendÿdcode�      �«goutcomeŸtorg.jruby.RubyStringgsuccessÿÿÿÿÿdMETAŸx�org.logstash.ConvertedMap¿dtypeŸtorg.jruby.RubyStringd_docÿgversionŸtorg.jruby.RubyStringe7.9.1ÿdbeatŸtorg.jruby.RubyStringjwinlogbeatÿjip_addressŸtorg.jruby.RubyStringk10.117.4.90ÿÿÿÿÿ   @580e728f08037f8b47d3261c243999e5b6f6def5c5465a4714cc8ce9ece64493   
elasticsearch  �TCould not index event to Elasticsearch. status: 400, action: ["index", {:_id=>nil, :_index=>"winlogbeat-7.9.1", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x6bf429fc>], response: {"index"=>{"_index"=>"winlogbeat-7.9.1-2020.09.23-000007", "_type"=>"_doc", "_id"=>"11-2wHQB5SvN9WG8XaPu", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id '11-2wHQB5SvN9WG8XaPu'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}c  �­ÿÿÿÿ Ý¿ñ   �2020-09-24T15:35:53.798Z  àŸqjava.util.HashMap¿dDATAŸx�org.logstash.ConvertedMap¿dhostŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringx�<redacted>.<redacted>.COMÿhhostnameŸtorg.jruby.RubyStringi<redacted>ÿÿÿfsourceŸx�org.logstash.ConvertedMap¿cgeoŸx�org.logstash.ConvertedMap¿ÿÿbipŸtorg.jruby.RubyStringeLOCALÿfdomainŸtorg.jruby.RubyStringgUnknownÿÿÿcecsŸx�org.logstash.ConvertedMap¿gversionŸtorg.jruby.RubyStringe1.5.0ÿÿÿduserŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringg<redacted>ÿfdomainŸtorg.jruby.RubyStringe<redacted>ÿÿÿclogŸx�org.logstash.ConvertedMap¿elevelŸtorg.jruby.RubyStringkinformationÿÿÿgmessageŸtorg.jruby.RubyStringy�™A session was disconnected from a Window Station.

Subject:
	Account Name:		<redacted>
	Account Domain:		<redacted>
	Logon ID:		0x1391B3

Session:
	Session Name:		Console

Additional Information:
	Client Name:		Unknown
	Client Address:		LOCAL


This event is generated when a user disconnects from an existing Terminal Services session, or when a user switches away from an existing desktop using Fast User Switching.ÿfwinlogŸx�org.logstash.ConvertedMap¿hevent_id�      �«gchannelŸtorg.jruby.RubyStringhSecurityÿjevent_dataŸx�org.logstash.ConvertedMap¿mClientAddressŸtorg.jruby.RubyStringeLOCALÿgLogonIDŸtorg.jruby.RubyStringh0x1391b3ÿkSessionNameŸtorg.jruby.RubyStringgConsoleÿjClientNameŸtorg.jruby.RubyStringgUnknownÿmAccountDomainŸtorg.jruby.RubyStringe<redacted>ÿkAccountNameŸtorg.jruby.RubyStringg<redacted>ÿÿÿmcomputer_nameŸtorg.jruby.RubyStringx�<redacted>.<redacted>.comÿmprovider_guidŸtorg.jruby.RubyStringx&{54849625-5478-4994-a5ba-3e3b0328c30d}ÿfopcodeŸtorg.jruby.RubyStringdInfoÿhkeywordsŸx�org.logstash.ConvertedListŸŸtorg.jruby.RubyStringmAudit SuccessÿÿÿdtaskŸtorg.jruby.RubyStringx�Other Logon/Logoff Eventsÿkactivity_idŸtorg.jruby.RubyStringx&{9fa4ca99-91c1-0003-d5ca-a49fc191d601}ÿirecord_id�     !EçcapiŸtorg.jruby.RubyStringkwineventlogÿmprovider_nameŸtorg.jruby.RubyStringx#Microsoft-Windows-Security-AuditingÿgprocessŸx�org.logstash.ConvertedMap¿fthreadŸx�org.logstash.ConvertedMap¿bid�      C�ÿÿcpid�      �œÿÿelogonŸx�org.logstash.ConvertedMap¿bidŸtorg.jruby.RubyStringh0x1391b3ÿÿÿÿÿdtagsŸx�org.logstash.ConvertedListŸŸtorg.jruby.RubyStringx�beats_input_codec_plain_appliedÿÿÿh@versiona1grelatedŸx�org.logstash.ConvertedMap¿duserŸtorg.jruby.RubyStringg<redacted>ÿÿÿeagentŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringi<redacted>ÿgversionŸtorg.jruby.RubyStringe7.9.1ÿlephemeral_idŸtorg.jruby.RubyStringx$b7816c03-6d97-4b0d-9b43-b1f54c7ab2d9ÿdtypeŸtorg.jruby.RubyStringjwinlogbeatÿbidŸtorg.jruby.RubyStringx$87e4e87f-fc64-4ed8-af12-6f552fe0a14dÿhhostnameŸtorg.jruby.RubyStringi<redacted>ÿÿÿj@timestampŸvorg.logstash.Timestampx�2020-09-24T15:35:50.861ZÿeeventŸx�org.logstash.ConvertedMap¿dkindŸtorg.jruby.RubyStringeeventÿfmoduleŸtorg.jruby.RubyStringhsecurityÿdtypeŸtorg.jruby.RubyStringcendÿfactionŸtorg.jruby.RubyStringtsession-disconnectedÿgcreatedŸtorg.jruby.RubyStringx�2020-09-24T15:35:52.481ZÿhproviderŸtorg.jruby.RubyStringx#Microsoft-Windows-Security-AuditingÿhcategoryŸtorg.jruby.RubyStringnauthenticationÿdcode�      �«goutcomeŸtorg.jruby.RubyStringgsuccessÿÿÿÿÿdMETAŸx�org.logstash.ConvertedMap¿dtypeŸtorg.jruby.RubyStringd_docÿgversionŸtorg.jruby.RubyStringe7.9.1ÿdbeatŸtorg.jruby.RubyStringjwinlogbeatÿjip_addressŸtorg.jruby.RubyStringl10.150.1.120ÿÿÿÿÿ   @580e728f08037f8b47d3261c243999e5b6f6def5c5465a4714cc8ce9ece64493   
elasticsearch  �TCould not index event to Elasticsearch. status: 400, action: ["index", {:_id=>nil, :_index=>"winlogbeat-7.9.1", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x3817283e>], response: {"index"=>{"_index"=>"winlogbeat-7.9.1-2020.09.23-000007", "_type"=>"_doc", "_id"=>"ovfBwHQBckcEdBsIssus", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id 'ovfBwHQBckcEdBsIssus'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}c  �Sÿÿÿÿ¶ý€   �2020-09-24T15:38:20.860Z  †Ÿqjava.util.HashMap¿dDATAŸx�org.logstash.ConvertedMap¿dhostŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringw<redacted>.<redacted>.COMÿhhostnameŸtorg.jruby.RubyStringh<redacted>ÿÿÿfsourceŸx�org.logstash.ConvertedMap¿cgeoŸx�org.logstash.ConvertedMap¿ÿÿbipŸtorg.jruby.RubyStringeLOCALÿfdomainŸtorg.jruby.RubyStringgUnknownÿÿÿcecsŸx�org.logstash.ConvertedMap¿gversionŸtorg.jruby.RubyStringe1.5.0ÿÿÿduserŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringg<redacted>ÿfdomainŸtorg.jruby.RubyStringe<redacted>ÿÿÿclogŸx�org.logstash.ConvertedMap¿elevelŸtorg.jruby.RubyStringkinformationÿÿÿgmessageŸtorg.jruby.RubyStringy��A session was reconnected to a Window Station.

Subject:
	Account Name:		<redacted>
	Account Domain:		<redacted>
	Logon ID:		0x12c77773

Session:
	Session Name:		Console

Additional Information:
	Client Name:		Unknown
	Client Address:		LOCAL

This event is generated when a user reconnects to an existing Terminal Services session, or when a user switches to an existing desktop using Fast User Switching.ÿfwinlogŸx�org.logstash.ConvertedMap¿hevent_id�      �ªgchannelŸtorg.jruby.RubyStringhSecurityÿjevent_dataŸx�org.logstash.ConvertedMap¿mClientAddressŸtorg.jruby.RubyStringeLOCALÿgLogonIDŸtorg.jruby.RubyStringj0x12c77773ÿkSessionNameŸtorg.jruby.RubyStringgConsoleÿjClientNameŸtorg.jruby.RubyStringgUnknownÿmAccountDomainŸtorg.jruby.RubyStringe<redacted>ÿkAccountNameŸtorg.jruby.RubyStringg<redacted>ÿÿÿmprovider_guidŸtorg.jruby.RubyStringx&{54849625-5478-4994-A5BA-3E3B0328C30D}ÿmcomputer_nameŸtorg.jruby.RubyStringw<redacted>.<redacted>.comÿfopcodeŸtorg.jruby.RubyStringdInfoÿhkeywordsŸx�org.logstash.ConvertedListŸŸtorg.jruby.RubyStringmAudit SuccessÿÿÿdtaskŸtorg.jruby.RubyStringx�Other Logon/Logoff EventsÿcapiŸtorg.jruby.RubyStringkwineventlogÿirecord_id�      �émprovider_nameŸtorg.jruby.RubyStringx#Microsoft-Windows-Security-AuditingÿgprocessŸx�org.logstash.ConvertedMap¿fthreadŸx�org.logstash.ConvertedMap¿bid�      �Ôÿÿcpid�      �0ÿÿelogonŸx�org.logstash.ConvertedMap¿bidŸtorg.jruby.RubyStringj0x12c77773ÿÿÿÿÿdtagsŸx�org.logstash.ConvertedListŸŸtorg.jruby.RubyStringx�beats_input_codec_plain_appliedÿÿÿh@versiona1grelatedŸx�org.logstash.ConvertedMap¿duserŸtorg.jruby.RubyStringg<redacted>ÿÿÿeagentŸx�org.logstash.ConvertedMap¿dnameŸtorg.jruby.RubyStringh<redacted>ÿgversionŸtorg.jruby.RubyStringe7.9.1ÿlephemeral_idŸtorg.jruby.RubyStringx$af57a7de-e59a-445f-9320-1e15053adb37ÿdtypeŸtorg.jruby.RubyStringjwinlogbeatÿbidŸtorg.jruby.RubyStringx$7c23fe19-c4f0-423d-94fd-5952dee5b471ÿhhostnameŸtorg.jruby.RubyStringh<redacted>ÿÿÿj@timestampŸvorg.logstash.Timestampx�2020-09-24T15:38:18.231ZÿeeventŸx�org.logstash.ConvertedMap¿dkindŸtorg.jruby.RubyStringeeventÿfmoduleŸtorg.jruby.RubyStringhsecurityÿdtypeŸtorg.jruby.RubyStringestartÿhcategoryŸtorg.jruby.RubyStringnauthenticationÿgcreatedŸtorg.jruby.RubyStringx�2020-09-24T15:38:20.268ZÿhproviderŸtorg.jruby.RubyStringx#Microsoft-Windows-Security-AuditingÿfactionŸtorg.jruby.RubyStringssession-reconnectedÿdcode�      �ªgoutcomeŸtorg.jruby.RubyStringgsuccessÿÿÿÿÿdMETAŸx�org.logstash.ConvertedMap¿dtypeŸtorg.jruby.RubyStringd_docÿgversionŸtorg.jruby.RubyStringe7.9.1ÿdbeatŸtorg.jruby.RubyStringjwinlogbeatÿjip_addressŸtorg.jruby.RubyStringk10.117.4.90ÿÿÿÿÿ   @580e728f08037f8b47d3261c243999e5b6f6def5c5465a4714cc8ce9ece64493   
elasticsearch  �TCould not index event to Elasticsearch. status: 400, action: ["index", {:_id=>nil, :_index=>"winlogbeat-7.9.1", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x7a24db34>], response: {"index"=>{"_index"=>"winlogbeat-7.9.1-2020.09.23-000007", "_type"=>"_doc", "_id"=>"DPnDwHQBckcEdBsI8YQb", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [source.ip] of type [ip] in document with id 'DPnDwHQBckcEdBsI8YQb'. Preview of field's value: 'LOCAL'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'LOCAL' is not an IP string literal."}}}}

andrewkroh added a commit to andrewkroh/beats that referenced this issue Sep 24, 2020
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes elastic#19627
@andrewkroh
Copy link
Member

I've opened #21325 to fix this.

@MakoWish
Copy link
Contributor Author

Thank you, @andrewkroh

andrewkroh added a commit that referenced this issue Sep 29, 2020
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes #19627
@zube zube bot added [zube]: Done and removed [zube]: Inbox labels Sep 29, 2020
@zube zube bot removed the [zube]: Done label Dec 28, 2020
andrewkroh added a commit to andrewkroh/beats that referenced this issue Jan 5, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes elastic#19627

(cherry picked from commit 8c992c5)
andrewkroh added a commit to andrewkroh/beats that referenced this issue Jan 5, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes elastic#19627

(cherry picked from commit 8c992c5)
andrewkroh added a commit that referenced this issue Jan 6, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes #19627

(cherry picked from commit 8c992c5)
andrewkroh added a commit that referenced this issue Jan 6, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes #19627

(cherry picked from commit 8c992c5)
@MakoWish
Copy link
Contributor Author

I am not sure how long the issue has been happening, but I just now noticed this appears to have regressed with the migration to Ingest Pipelines for Winlogbeat instead of local parsing. As a temporary workaround, I have modified the winlogbeat-8.3.3-security Ingest Pipeline to add a correction for the values "LOCAL" or "Unknown":

//ClientAddress to source.ip and related.ip
if (ctx?.winlog?.event_data?.ClientAddress != null &&
  ctx.winlog.event_data.ClientAddress != "-") {
// Fix source.ip:LOCAL regression https://github.com/elastic/beats/issues/19627
if (ctx?.winlog?.event_data?.ClientAddress == "LOCAL" ||
    ctx?.winlog?.event_data?.ClientAddress == "Unknown") {
  ctx.winlog.event_data.ClientAddress="127.0.0.1";
}
if (ctx?.source == null) {
  HashMap hm = new HashMap();
  ctx.put("source", hm);
}
if (ctx?.related == null) {
  HashMap hm = new HashMap();
  ctx.put("related", hm);
}
if (ctx?.related?.ip == null) {
  ArrayList al = new ArrayList();
  ctx.related.put("ip", al);
}
ctx.source.put("ip", ctx.winlog.event_data.ClientAddress);
if (!ctx.related.ip.contains(ctx.winlog.event_data.ClientAddress)) {
  ctx.related.ip.add(ctx.winlog.event_data.ClientAddress);
}
}

Can someone on the Elastic side address this with a permanent fix?

Eric

@MakoWish
Copy link
Contributor Author

MakoWish commented Jan 12, 2023

Opened #34252 for this. This is my first attempt at contributing anything to GH. Can someone give a quick walkthrough on how to push the changes? I forked the Beats repo and made the changes there, but not sure how to push this up for review. Changes here to my fork here.

@MakoWish
Copy link
Contributor Author

I think I got it figured out. PR is just pending approval. I have signed a Contributor Agreement, but it still says that I have not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment