Add Kerberos Events - Other Logon Events - ECS event Categories and Types #17517

janniten · 2020-04-06T13:51:53Z

Added Kerberos Tickets Events (4768,4769,4770,4771,4776), aditional logon events (4778,4779,4964), sensitive privilege use events (4673,4674), service and task scheduler events (4697,4698,4699,4700,4701,4702)

Event categories, types and outcome were aligned to the ones defined in ECS 1.5

Other minor fixes/improvements introduced

New Events
The new events are commonly used when analyzing lateral movements
Sources:
https://www.jpcert.or.jp/english/pub/sr/20170612ac-ir_research_en.pdf
https://www.sans.org/security-resources/posters/dfir/windows-forensic-analysis-170
https://www.sans.org/security-resources/posters/dfir/hunt-evil-165
https://www.sans.org/security-resources/posters/dfir/windows-forensics-evidence-of-75

Event	Description
4673	A privileged service was called.
4674	An operation was attempted on a privileged object.
4697	A service was installed in the system.
4698	A scheduled task was created.
4699	A scheduled task was deleted.
4700	A scheduled task was enabled.
4701	A scheduled task was disabled.
4702	A scheduled task was updated.
4768	A Kerberos authentication ticket TGT was requested.
4769	A Kerberos service ticket was requested.
4770	A Kerberos service ticket was renewed.
4771	Kerberos pre-authentication failed.
4776	The computer attempted to validate the credentials for an account.
4778	A session was reconnected to a Window Station.
4779	A session was disconnected from a Window Station.
4964	Special groups have been assigned to a new logon.

Event Categories and Types

ECS 1.5 define which are the values allowed for event categories and types.
https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-category.html
https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-type.html

Modifications have been introduced in order to adjust the events to fall into allowed categories/types.
In order to add flexibility the function addActionDesc was replace for a more general function, the addEventFields. This function completes the category (event.category), the primary type (event.type) and the action description (event.action).
Aditional types can be attached to the event via AppendTo function.
For example, for Evt 4719 the addEventFields sets the event.category="iam", event.type="admin" but because it is also a change thet event.type=change is also added via AppendTo in the specific function that process the event
Having more than one value in the event.type, allow us to detect, for example, this type of activity

 event.category:iam AND event.type:admin AND event.type:change

In this way event categories/types can be easily modified according to futher ECS changes.

For all events the evet.outcome was also aligned to the specification of ECS 1.5
https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-outcome.html

Fixes/Improvements

Event 4648:
Events 4648 and 4624 were processed by the same function, but the winlog.logon.id field needs to be copied from different source field in the case of evt 4648
For evt 4624 the winlog.logon.id comes from winlog.data.TargetLogonID whereas that for evt 4648 should be copied from winlog.data.SubjectLogonID
The processing for those events were decoupled.
In the Evt 4648 we have now the winlog.logon.id of the user logging in with the explicit credentials and thus be able to correlate with the original winlog.logon.id. See example below

Event 4625:
In this event the Add(copyTargetUserLogonId) was replaced for the .Add(copySubjectUserLogonId). The relevant LogonID information is in the winlog.event_data.SubjectLogonId and was obtain from winlog.event_dataTargetLogonId
Normaly the winlog.event_data.SubjectLogonId is 0x0 but when one logged-in user tries to run as a different user and fails to, the LogonID is the one of the logged-in user
In this way we can track all the activity, right from the original connection. See example below
Events 4728,4729,4732,4733,4743,4746,4747,4751,4752,4756,4757,4761,4762 :
Extracted user from winlog.event_data.MemberName in order to be added to related.user field
Events 4688 and 4689: winlog.logon.id field was missing. The .Add(copySubjectUserLogonId) was added.
Event 4688: Modified to not to add related.user when winlog.event_data.TargetUserName is "-"
Event 4672: add event.outcome. Aligned to ECS 1.5
Event 4647: Missing event.action
Event 4731 and 1102: Wrong event.action
copyTargetUser was modify in order to parse properly the user name in Kerberos Events
Added missing codes to logonFailureStatus table in order to map failure codes from Event 4776
addAuthSuccess and addAuthFailed removed and replaced for addEventOutcome and addEventFields functions.
Enrichment information sources. For all tables with data used to enrich events the source of that information was added.

Future Work

From the discussion [ECS] Multiple users in an event proposal ecs#809 will be (probably) modifications to perform in order to adapt events with mutiples users
Policy Modifications Events

elasticmachine · 2020-04-06T13:51:55Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

elasticmachine · 2020-04-06T13:51:55Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

janniten · 2020-04-06T13:56:29Z

@andrewkroh @leehinman some new events added, adjustment to ECS event categories and types and some fixes/improvements

elasticmachine · 2020-04-07T07:46:48Z

Pinging @elastic/siem (Team:SIEM)

leehinman

@janniten This is awesome.

I did get some errors when I tried to make the golden files.

    --- FAIL: TestSecurity/security-windows2012_4770.evtx (0.02s)
        testing_windows.go:70: failed in processor.javascript: failed in process function: TypeError: Cannot read property '2' of undefined at ..\config\winlogbeat-security.js:1356:38(17) while processing event:
...

It doesn't look like you don't have events 4770, 4771 in the eventActionTypes object. Could you add those?

x-pack/winlogbeat/module/security/_meta/docs.asciidoc

x-pack/winlogbeat/module/security/config/winlogbeat-security.js

janniten · 2020-04-07T16:35:03Z

@leehinman I have implemented all the changes you have suggested. Thank you!
Please let me know If I need to perform further modifications

leehinman

hit a different error with 4771

PS C:\Gopath\src\github.com\elastic\beats\x-pack\winlogbeat\module\security\test> go test -update .\security_windows_test.go
--- FAIL: TestSecurity (1.30s)
    --- FAIL: TestSecurity/security-windows2012_4771.evtx (0.03s)
        testing_windows.go:70: failed in processor.javascript: failed in process function: TypeError: Cannot read property 'toLowerCase' of undefined at ..\config\winlogbeat-security.js:1481:27(7) while processing event:
            {
              "error": {
                "message": "TypeError: Cannot read property 'toLowerCase' of undefined at ..\\config\\winlogbeat-security.js:1481:27(7)"

janniten · 2020-04-07T18:23:38Z

hit a different error with 4771

PS C:\Gopath\src\github.com\elastic\beats\x-pack\winlogbeat\module\security\test> go test -update .\security_windows_test.go
--- FAIL: TestSecurity (1.30s)
    --- FAIL: TestSecurity/security-windows2012_4771.evtx (0.03s)
        testing_windows.go:70: failed in processor.javascript: failed in process function: TypeError: Cannot read property 'toLowerCase' of undefined at ..\config\winlogbeat-security.js:1481:27(7) while processing event:
            {
              "error": {
                "message": "TypeError: Cannot read property 'toLowerCase' of undefined at ..\\config\\winlogbeat-security.js:1481:27(7)"

Correct, Event 4771 does not have the EcnryptionType Fields. Sorry
Fixed.

leehinman

Looking good. I was able to produce golden files on win2012, win2016 & win2019. After the fix below do you want me to push them to your branch?

Also, could you add an entry to CHANGELOG.next.asciidoc ?

leehinman

event.action shouldn't be an array

x-pack/winlogbeat/module/security/config/winlogbeat-security.js

janniten · 2020-04-08T16:40:47Z

Looking good. I was able to produce golden files on win2012, win2016 & win2019. After the fix below do you want me to push them to your branch?

Yes, please. Thank you!

leehinman · 2020-04-09T15:00:03Z

jenkins, test this

leehinman · 2020-04-15T21:06:53Z

@janniten we need to rebase off master to get rid of the merge conflicts. Would you like me to do that or do you want to?

janniten · 2020-04-15T21:16:12Z

@leehinman, Go ahead! Thank you 😊

… Aligment

…nt type/category/description - Spacing fixed - Ticket Encryption Options Normalized

leehinman · 2020-04-16T20:12:56Z

jenkins, test this

leehinman · 2020-04-16T20:14:43Z

run tests

andrewkroh

Another great contribution. Thanks!

Would you mind taking a pass through the JS file to ensure consistent formatting. I should probably add some kind of linting check to the project for these pipelines to make this easier.

spaces between function args
spaces around = during assignments.
lower camel case naming

andrewkroh · 2020-04-20T14:31:48Z

x-pack/winlogbeat/module/security/test/testdata/security-windows2012_4673.evtx.golden.json

+      "level": "information"
+    },
+    "process": {
+      "executable": "C:\\Windows\\System32\\lsass.exe",


We can populate process.command_line here too.

Hi @andrewkroh , I did all modifications about lower camel case, spaces in assignments and between function args. Now testing the changes in my environment and then I'll commit the changes soon.
I have a doubt regarding to populate the process.command_line in event 4673.
I'm not able to see that information in the event, as for example in event 4688. Am I missing something?

andrewkroh · 2020-04-20T14:36:55Z

x-pack/winlogbeat/module/security/config/winlogbeat-security.js

-        [0x800000, "PASSWORD_EXPIRED"],
-        [0x1000000, "TRUSTED_TO_AUTH_FOR_DELEGATION"],
-        [0x4000000, "PARTIAL_SECRETS_ACCOUNT"],
+    var uac_flags = [


The rest of the file appears to use lower camel case.

Suggested change

var uac_flags = [

var uacFlags = [

andrewkroh · 2020-04-20T14:41:25Z

x-pack/winlogbeat/module/security/config/winlogbeat-security.js

+        if (! uac_list) {
+            return;
+        }
+        evt.Put("winlog.event_data.UserAccountControl",uac_list);


Suggested change

evt.Put("winlog.event_data.UserAccountControl",uac_list);

evt.Put("winlog.event_data.UserAccountControl", uac_list);

For formatting consistency and readability it would be good to consitently use a space between args.

andrewkroh · 2020-04-20T14:45:46Z

x-pack/winlogbeat/module/security/config/winlogbeat-security.js

            return;
        }
-        if (!auditDescription[subcategoryGuid]) {
+        var tkt_code=parseInt(code,16).toString(2);


Suggested change

var tkt_code=parseInt(code,16).toString(2);

var tkt_code = parseInt(code,16).toString(2);

janniten · 2020-04-21T10:04:19Z

Another great contribution. Thanks!

Would you mind taking a pass through the JS file to ensure consistent formatting. I should probably add some kind of linting check to the project for these pipelines to make this easier.

spaces between function args

spaces around = during assignments.

lower camel case naming

Hi @andrewkroh ; I'll pass through the JS in order to identify the points you have mentioned, my pleasure :)

But I need a little bit of help in order to locally update my PR (sorry, I'm not a developer, just a security girl :) ).
When I try to rebase, this error occurs (see attached). I tried few things but with luck
@leehinman can you help me?

Thank you!
error.txt

andrewkroh · 2020-04-21T10:55:08Z

Since the branch was updated you'll probably just want to reset your local branch to match the current state of the PR before making any new changes. This will wipe out any changes to the local branch that are not in this PR.

git fetch --all
git reset --hard origin/logon-kbr-evts

janniten · 2020-04-21T11:48:57Z

Since the branch was updated you'll probably just want to reset your local branch to match the current state of the PR before making any new changes. This will wipe out any changes to the local branch that are not in this PR.
git fetch --all
git reset --hard origin/logon-kbr-evts

Thank you! It worked. I'll be reviewing the and submitting the changes today

elasticmachine · 2020-05-03T13:33:23Z

💚 Build Succeeded

Expand to view the summary

Build stats

Build Cause: [Pull request Add Kerberos Events - Other Logon Events - ECS event Categories and Types #17517 updated]
Start Time: 2020-05-03T12:38:09.382+0000
Duration: 55 min 9 sec (3309445)
Commit: 301c273

Test stats 🧪

Test	Results
Failed	0
Passed	261
Skipped	1
Total	262

…ypes (elastic#17517) Add support for event IDs 4673, 4674, 4697, 4698, 4699, 4700, 4701, 4702, 4768, 4769, 4770, 4771, 4776, 4778, 4779, 4964 to the Winlogbeat Security module. Co-authored-by: Lee E. Hinman <lee.e.hinman@elastic.co> (cherry picked from commit 955bc46)

…ypes (#17517) (#18174) Add support for event IDs 4673, 4674, 4697, 4698, 4699, 4700, 4701, 4702, 4768, 4769, 4770, 4771, 4776, 4778, 4779, 4964 to the Winlogbeat Security module. Co-authored-by: Lee E. Hinman <lee.e.hinman@elastic.co> (cherry picked from commit 955bc46) Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>

…18775) This PR adds two new dashboards related to events added in PRs (#12906, #14299, #15217, #17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>

…lastic#18775) This PR adds two new dashboards related to events added in PRs (elastic#12906, elastic#14299, elastic#15217, elastic#17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (elastic#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>

…lastic#18775) This PR adds two new dashboards related to events added in PRs (elastic#12906, elastic#14299, elastic#15217, elastic#17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (elastic#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> (cherry picked from commit 7b9c535)

…18775) (#22598) This PR adds two new dashboards related to events added in PRs (#12906, #14299, #15217, #17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> (cherry picked from commit 7b9c535) Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>

janniten requested a review from a team as a code owner April 6, 2020 13:51

janniten changed the title ~~Add Kerberos Event - Other Logon Events - ECS event Categories and Types~~ Add Kerberos Events - Other Logon Events - ECS event Categories and Types Apr 6, 2020

andresrc added [zube]: Inbox Team:SIEM labels Apr 7, 2020

leehinman requested changes Apr 7, 2020

View reviewed changes

janniten mentioned this pull request Apr 8, 2020

[Winlogbeat] Update Winlogbeat security module to ECS 1.4 #17528

Closed

leehinman requested changes Apr 8, 2020

View reviewed changes

x-pack/winlogbeat/module/security/config/winlogbeat-security.js Outdated Show resolved Hide resolved

janniten and others added 7 commits April 16, 2020 12:59

Add Kerberos Event - Other Logon Events - ECS event Category and Type…

461c647

… Aligment

AppendTo function in addEventFields - Added Missing 4770 and 4771 eve…

20e7ced

…nt type/category/description - Spacing fixed - Ticket Encryption Options Normalized

Fix for Evt 4771 Encryption Type

91a6177

Fixed evt.Put for event action - CHANGELOG.next.asciidoc Completed

e4e1e17

Add golden files & tidy white space in winlogbeat-security.js

c91c8d4

fixes for rebase

2c8cb23

fix CHANGELOG.next.asciidoc

bd8c2e3

leehinman approved these changes Apr 16, 2020

View reviewed changes

update documentation

6e4c6ef

andrewkroh reviewed Apr 20, 2020

View reviewed changes

andrewkroh added enhancement Winlogbeat labels Apr 23, 2020

Fix: Use Lower Case Camel and spacing between parameters and assignments

301c273

andrewkroh merged commit 955bc46 into elastic:master May 4, 2020

andrewkroh added the needs_backport PR is waiting to be backported to other branches. label May 4, 2020

zube bot added [zube]: Done and removed [zube]: Inbox needs_backport PR is waiting to be backported to other branches. labels May 4, 2020

andrewkroh mentioned this pull request May 4, 2020

Cherry-pick #17517 to 7.x: Add Kerberos Events - Other Logon Events - ECS event Categories and Types #18174

Merged

andrewkroh added the v7.8.0 label May 4, 2020

janniten mentioned this pull request May 27, 2020

Winlogbeat Security new dashboards - Older dashboards improvements #18775

Merged

2 tasks

janniten mentioned this pull request Jul 23, 2020

[Winlogbeat] Event IDs to add to Security Module #16334

Closed

andrewkroh mentioned this pull request Nov 16, 2020

Cherry-pick #18775 to 7.x: Winlogbeat Security new dashboards - Older dashboards improvements #22598

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Kerberos Events - Other Logon Events - ECS event Categories and Types #17517

Add Kerberos Events - Other Logon Events - ECS event Categories and Types #17517

janniten commented Apr 6, 2020

elasticmachine commented Apr 6, 2020

elasticmachine commented Apr 6, 2020

janniten commented Apr 6, 2020

elasticmachine commented Apr 7, 2020

leehinman left a comment

janniten commented Apr 7, 2020

leehinman left a comment

janniten commented Apr 7, 2020

leehinman left a comment

leehinman left a comment

janniten commented Apr 8, 2020

leehinman commented Apr 9, 2020

leehinman commented Apr 15, 2020

janniten commented Apr 15, 2020

leehinman commented Apr 16, 2020

leehinman commented Apr 16, 2020

andrewkroh left a comment

andrewkroh Apr 20, 2020

janniten Apr 22, 2020

andrewkroh Apr 20, 2020

andrewkroh Apr 20, 2020

andrewkroh Apr 20, 2020

janniten commented Apr 21, 2020

andrewkroh commented Apr 21, 2020

janniten commented Apr 21, 2020

elasticmachine commented May 3, 2020

Build stats

Test stats 🧪

	evt.Put("winlog.event_data.UserAccountControl",uac_list);
	evt.Put("winlog.event_data.UserAccountControl", uac_list);

	var tkt_code=parseInt(code,16).toString(2);
	var tkt_code = parseInt(code,16).toString(2);

Add Kerberos Events - Other Logon Events - ECS event Categories and Types #17517

Add Kerberos Events - Other Logon Events - ECS event Categories and Types #17517

Conversation

janniten commented Apr 6, 2020

elasticmachine commented Apr 6, 2020

elasticmachine commented Apr 6, 2020

janniten commented Apr 6, 2020

elasticmachine commented Apr 7, 2020

leehinman left a comment

Choose a reason for hiding this comment

janniten commented Apr 7, 2020

leehinman left a comment

Choose a reason for hiding this comment

janniten commented Apr 7, 2020

leehinman left a comment

Choose a reason for hiding this comment

leehinman left a comment

Choose a reason for hiding this comment

janniten commented Apr 8, 2020

leehinman commented Apr 9, 2020

leehinman commented Apr 15, 2020

janniten commented Apr 15, 2020

leehinman commented Apr 16, 2020

leehinman commented Apr 16, 2020

andrewkroh left a comment

Choose a reason for hiding this comment

andrewkroh Apr 20, 2020

Choose a reason for hiding this comment

janniten Apr 22, 2020

Choose a reason for hiding this comment

andrewkroh Apr 20, 2020

Choose a reason for hiding this comment

andrewkroh Apr 20, 2020

Choose a reason for hiding this comment

andrewkroh Apr 20, 2020

Choose a reason for hiding this comment

janniten commented Apr 21, 2020

andrewkroh commented Apr 21, 2020

janniten commented Apr 21, 2020

elasticmachine commented May 3, 2020

💚 Build Succeeded

Build stats

Test stats 🧪