Winlogbeat - include structured event data with each event

[andrewkroh]

Implements #1053

Changes

• Add additional data to the events published by Winlogbeat. The new fields are activity_id,
  event_data, keywords, opcode, process_id, provider_guid, related_activity_id,
  task, thread_id, user_data. and version. Examples of the new events can be seen here.
• The message_inserts field was replaced with the event_data field
• The category field was renamed to task to better align with the Windows Event Log API naming

Benefits

The benefit to this change is that Winlogbeat now provides the data from the event log message in a structured format. For example, for a user login event the text from the message may be

Subject:
    Security ID:        S-1-5-21-3541430928-2051711210-1391384369-1001
    Account Name:        vagrant
    Account Domain:        VAGRANT-2012-R2
    Logon ID:        0x837F2   

and to make use of this message most users would have previously used Logstash to grok the key/value data. Now, included in the JSON event is the same data in a structured format:

  "event_data": {
    "LogonType": "8",
    "TargetDomainName": "VAGRANT-2012-R2",
    "TargetLogonId": "0x837f2",
    "TargetUserName": "vagrant",
    "TargetUserSid": "S-1-5-21-3541430928-2051711210-1391384369-1001"
  }

What's Left to Do in a Separate PR

• Due to the refactoring which simplified how event data was retrieved from the Windows Event Log API, we lost the ability to render partial events when errors occur. I need to improve the code that interfaces to Windows so that it falls back to rendering the event without the message so that we get parts of the event without the message. Implemented in Report event log records when a rendering error occurs #1180

[ruflin]

@andrewkroh Unrelated comment: I would recommend to most of the content you put into the PR description to also put it into the commit message. The reason is PR's are Github related, if someone only looks at commits for example with blame, he doesn't see all the additional info.

[ruflin]

The type in elasticsearch seems to be "integer" an not "int": https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html#number Not sure if both works. We use int also in all other fields.yml so we would have to change it everywhere.

[andrewkroh]

I will fix these is a separate PR. This will change the template and mapping for these fields (long -> integer) so I want to make sure there are no unintended consequences.

[ruflin]

LGTM

[andrewkroh]

@ruflin I agree and I amended the commit message to include more of the details I wrote in the PR. (And btw, github does associate commits to pull requests when you browse by commit. It provides a rather small link to the PR.)

[ruflin]

@andrewkroh Found the link. I never saw that one before. Good to know.