Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Group Management Events - Add NewUAC Description for User Management Events #14299

Merged
merged 12 commits into from
Dec 17, 2019
Merged

Add Group Management Events - Add NewUAC Description for User Management Events #14299

merged 12 commits into from
Dec 17, 2019

Conversation

janniten
Copy link
Contributor

@janniten janniten commented Oct 29, 2019

Added Group Management Events and Events 4798 and 4799 (User and Group enumeration)

Event Description
4727 A security-enabled global group was created.
4728 A member was added to a security-enabled global group.
4729 A member was removed from a security-enabled global group.
4730 A security-enabled global group was deleted.
4731 A security-enabled local group was created
4732 A member was added to a security-enabled local group.
4733 A member was removed from a security-enabled local group.
4734 A security-enabled local group was deleted.
4735 A security-enabled local group was changed.
4737 A security-enabled global group was changed.
4754 A security-enabled universal group was created.
4755 A security-enabled universal group was changed.
4756 A member was added to a security-enabled universal group.
4757 A member was removed from a security-enabled universal group.
4758 A security-enabled universal group was deleted.
4764 A group's type was changed.
4798 A user's local group membership was enumerated.
4799 A security-enabled local group membership was enumerated.

In order to map correclty this event a new field group.domain was added to the ECS group schema
(elastic/ecs#547)

Added addUACDescription function in order to translate to a human readable form the flags in winlog.event_data.NewUacValue field. A new field winlog.event_data.NewUACList is created and contains a list of decoded flags from the hex value in winlog.event_data.NewUACList
For example
winlog.event_data.NewUacValue -> 0x15 is translated to winlog.event_data.NewUACList -> SCRIPT,LOCKOUT
Also converts the winlog.event_data.UserAccountControl to a list of values

Cosmetic change: fix the order of events 4767 and 4781 and some space fixing

Ingesting this events we can have information about group who performed group changes and what the changes are.

Group_Managment - Kibana

@janniten janniten requested a review from a team as a code owner October 29, 2019 10:46
@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

1 similar comment
@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@janniten
Copy link
Contributor Author

@andrewkroh
Group Management Events and some improvements to User Management Events added in this PR

Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a reviewer for Beats code, just jumping in to look at the values and field names.

Question for Beats maintainers: Is there a way to have test data for this in the repo? Similar to Filebeat module's test fixtures.

@@ -27,15 +55,34 @@ var security = (function () {
"4720": "added-user-account",
"4722": "enabled-user-account",
"4723": "changed-password",
"4724": "reset-password",
"4724": "reseted-password",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually think the initial value of "reset-password" was better. I don't think "reseted" is an english word :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed :)

@andrewkroh andrewkroh self-requested a review October 31, 2019 16:52
@andrewkroh
Copy link
Member

andrewkroh commented Oct 31, 2019

This dashboard looks great. Would you like to add that to the PR?

.\winlogbeat.exe export dashboard -id [uuid-of-the-dashboard-taken-from-the-url] > my-dashboard.json

(I haven't tried this on Windows, but I think it should work. This assumes Kibana is a localhost:5601.)

@janniten
Copy link
Contributor Author

janniten commented Nov 4, 2019

This dashboard looks great. Would you like to add that to the PR?

.\winlogbeat.exe export dashboard -id [uuid-of-the-dashboard-taken-from-the-url] > my-dashboard.json

(I haven't tried this on Windows, but I think it should work. This assumes Kibana is a localhost:5601.)
Yes! I'll add the dashboard to the PR today :)

@janniten
Copy link
Contributor Author

janniten commented Nov 5, 2019

@andrewkroh I have already exported the dashboard in a .json.
Shall I put the exported dashboards in the folder beats/winlogbeat/_meta/kibana/7/dashboard ?
I have included also a similar dashboard for user management events.

I'll wait for your response and then I'll update the PR.

@andrewkroh
Copy link
Member

Shall I put the exported dashboards in the folder beats/winlogbeat/_meta/kibana/7/dashboard ?

Since these dashboards are dependent on the transformations done in the module I think they belong in the module dir at x-pack/winlogbeat/module/security/_meta/kibana/7/dashboard. Thanks

@janniten
Copy link
Contributor Author

janniten commented Nov 5, 2019

Done!
I put in the PR dashboards for User Management Events (#13530) and for Group Management Events.
In both cases I have included two versions: one using Visual Builder metrics that are nicer & colorful (but may require to adjust the interval size in the Panel Options according to your own needs) and one with normal metric visualizations that can be used without adjusting any parameter.

@janniten
Copy link
Contributor Author

@andrewkroh, It is not clear for me which is the error and If I have to correct something
It seems some problem with the Json related to the new dashboards.
Shall I remove the dashboards in order to unblock the PR?
Once this PR is closed I'm preparing a new one with the events of computer management (4741,4742 and 4743) and the events related to audit change/deletion/errors (1100,1102,1104,1105,1108 and 4719)

Thank you,
Regards
Ana

Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janniten . Thank you this is really helpful. I did have a few whitespace changes, can you take a look? Also I generated new golden.json files to go with the new test data you added. Would it be OK if I added those to the PR?

@janniten
Copy link
Contributor Author

Hi @leehinman
I have fixed the spacing problems that were found.
Please feel free to add the golden files and ask me any further modification.
Thank you!

 - fix whitespace in winlogbeat-security.js to make diffs cleaner
@leehinman
Copy link
Contributor

jenkins, test this

 - used libbeat/scripts/unpack_dashboards.py --transform decode
@janniten
Copy link
Contributor Author

janniten commented Dec 5, 2019

@janniten I got the golden files added and updated the dashboards to pass the tests. Can you double check the dashboards to make sure I didn't break them?

@leehinman, Maybe I can remove the dashboards so the PR can be merge and create a separate PR for the dashboards. What do you think?

@leehinman
Copy link
Contributor

@janniten I got the golden files added and updated the dashboards to pass the tests. Can you double check the dashboards to make sure I didn't break them?

@leehinman, Maybe I can remove the dashboards so the PR can be merge and create a separate PR for the dashboards. What do you think?

@janniten Let me see if I can test the dashboard, @andrewkroh gave me some pointers I'd like to try out.

@janniten
Copy link
Contributor Author

@janniten Let me see if I can test the dashboard, @andrewkroh gave me some pointers I'd like to try out.

Hi @leehinman, Can I help/do something more?

@leehinman
Copy link
Contributor

@janniten I haven't (yet) figured out how to get the dashboards working. If you are still willing maybe we should split the PR. I'll keep working on the dashboards.

@janniten
Copy link
Contributor Author

@janniten I haven't (yet) figured out how to get the dashboards working. If you are still willing maybe we should split the PR. I'll keep working on the dashboards.

@leehinman I removed the dashboards. I think now is ok.
Once merged I'll create two more PRs: one with the dashboards and one for others windows events I've been working

@leehinman
Copy link
Contributor

jenkins, test this

@janniten
Copy link
Contributor Author

jenkins, test this

@leehinman , shall I modify something? CI Build failed

@leehinman leehinman merged commit 8e31628 into elastic:master Dec 17, 2019
@leehinman leehinman added the needs_backport PR is waiting to be backported to other branches. label Dec 17, 2019
leehinman pushed a commit to leehinman/beats that referenced this pull request Dec 17, 2019
… User Management Events (elastic#14299)

* Added Group Management Events
* Added User and Group Enumeration
* Added New UAC Description

(cherry picked from commit 8e31628)
@leehinman leehinman added v7.6.0 and removed needs_backport PR is waiting to be backported to other branches. labels Dec 17, 2019
@janniten janniten deleted the evt-group-mgmt branch December 18, 2019 09:10
leehinman added a commit that referenced this pull request Jan 2, 2020
… User Management Events (#14299) (#15153)

* Added Group Management Events
* Added User and Group Enumeration
* Added New UAC Description

(cherry picked from commit 8e31628)

Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>
andrewkroh added a commit that referenced this pull request Feb 5, 2020
…module (#15217)

Added Audit and Log Management related events, Computer Object Management Events, Distribution Groups Events. Changed user.name field for user management events and related.user mapping.

New Events

Due to that Windows events are the source of information for Winlogbeat the events 1100, 1102, 1104, 1105, 1108 and 4719 has been added in order to monitor changes in the audit policy configuration, log deletion and other failures in the log subsystem.

For event 4719, a human readable description was added in order to know which setting was modified (winlog.event_data.SubCategory) and to which value (winlog.event_data.AuditPolicyChangesDescription).

Distribution Groups (Security-Disabled) Management Events were added. Those events are processed in the same way and with the same function that Security Groups (#14299). In order to add information about the nature of the group being managed the type (Security-Disabled/Security-Enabled) and scope (Local,Global,Universal) where added as winlog.group.type and winlog.group.scope.

ComputerObject Management events were also added.

Changes to ECS mappings

In elastic/ecs#678 and elastic/ecs#589 we have been discussing how n-ary relationship between users in an event should be named and mapping into ECS. In #13530 winlog.event_data.TargetUserName has been mapped to user.name but from the reasons exposed in elastic/ecs#678 and elastic/ecs#589 the mapping winlog.event_data.SubjectUserName -> user.name is more appropriate. This mapping was changed.

Also, with the adding of related fields in ECS 1.3 and specifically the related.user field (elastic/ecs#694) all the user names appearing in one event were mapped to the related user events. Every time a SubjectUserName or TargetUserName is copied also is added to the related.user field, as well as other users appearing in the event.

Event test data were added for all events with the exception of event 1108 which I was not able to reproduce.

Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com>
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
@janniten
Copy link
Contributor Author

@leehinman one question. Events added in this PR are not listed in the release notes
https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-7.6.0.html

Do you know why?
Thank you

@leehinman
Copy link
Contributor

@leehinman one question. Events added in this PR are not listed in the release notes
https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-7.6.0.html

Do you know why?
Thank you

@janniten I'm sorry, this is my fault. I didn't catch that there wasn't a Changelog entry, and the info in the release notes is populated from the Changelog. I did check and the code is in the release.

    [Winlogbeat] Add Group Management Events - Add NewUAC Description for User Management Events (#14299) (#15153)

    * Added Group Management Events
    * Added User and Group Enumeration
    * Added New UAC Description

    (cherry picked from commit 8e31628955f89ab550e33a1ae1cc750b850a29ab)

    Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>

andrewkroh pushed a commit to andrewkroh/beats that referenced this pull request Mar 18, 2020
…module (elastic#15217)

Added Audit and Log Management related events, Computer Object Management Events, Distribution Groups Events. Changed user.name field for user management events and related.user mapping.

New Events

Due to that Windows events are the source of information for Winlogbeat the events 1100, 1102, 1104, 1105, 1108 and 4719 has been added in order to monitor changes in the audit policy configuration, log deletion and other failures in the log subsystem.

For event 4719, a human readable description was added in order to know which setting was modified (winlog.event_data.SubCategory) and to which value (winlog.event_data.AuditPolicyChangesDescription).

Distribution Groups (Security-Disabled) Management Events were added. Those events are processed in the same way and with the same function that Security Groups (elastic#14299). In order to add information about the nature of the group being managed the type (Security-Disabled/Security-Enabled) and scope (Local,Global,Universal) where added as winlog.group.type and winlog.group.scope.

ComputerObject Management events were also added.

Changes to ECS mappings

In elastic/ecs#678 and elastic/ecs#589 we have been discussing how n-ary relationship between users in an event should be named and mapping into ECS. In elastic#13530 winlog.event_data.TargetUserName has been mapped to user.name but from the reasons exposed in elastic/ecs#678 and elastic/ecs#589 the mapping winlog.event_data.SubjectUserName -> user.name is more appropriate. This mapping was changed.

Also, with the adding of related fields in ECS 1.3 and specifically the related.user field (elastic/ecs#694) all the user names appearing in one event were mapped to the related user events. Every time a SubjectUserName or TargetUserName is copied also is added to the related.user field, as well as other users appearing in the event.

Event test data were added for all events with the exception of event 1108 which I was not able to reproduce.

Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com>
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
(cherry picked from commit e624aef)
andrewkroh added a commit that referenced this pull request Mar 18, 2020
…ent Events - ECS related.user field mapping (#17090)

Added Audit and Log Management related events, Computer Object Management Events, Distribution Groups Events. Changed user.name field for user management events and related.user mapping.

New Events

Due to that Windows events are the source of information for Winlogbeat the events 1100, 1102, 1104, 1105, 1108 and 4719 has been added in order to monitor changes in the audit policy configuration, log deletion and other failures in the log subsystem.

For event 4719, a human readable description was added in order to know which setting was modified (winlog.event_data.SubCategory) and to which value (winlog.event_data.AuditPolicyChangesDescription).

Distribution Groups (Security-Disabled) Management Events were added. Those events are processed in the same way and with the same function that Security Groups (#14299). In order to add information about the nature of the group being managed the type (Security-Disabled/Security-Enabled) and scope (Local,Global,Universal) where added as winlog.group.type and winlog.group.scope.

ComputerObject Management events were also added.

Changes to ECS mappings

In elastic/ecs#678 and elastic/ecs#589 we have been discussing how n-ary relationship between users in an event should be named and mapping into ECS. In #13530 winlog.event_data.TargetUserName has been mapped to user.name but from the reasons exposed in elastic/ecs#678 and elastic/ecs#589 the mapping winlog.event_data.SubjectUserName -> user.name is more appropriate. This mapping was changed.

Also, with the adding of related fields in ECS 1.3 and specifically the related.user field (elastic/ecs#694) all the user names appearing in one event were mapped to the related user events. Every time a SubjectUserName or TargetUserName is copied also is added to the related.user field, as well as other users appearing in the event.

Event test data were added for all events with the exception of event 1108 which I was not able to reproduce.

Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com>
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>

(cherry picked from commit e624aef)
andrewkroh added a commit that referenced this pull request Jun 4, 2020
…18775)

This PR adds two new dashboards related to events added in PRs (#12906, #14299, #15217, #17517) and implements some improvements to existing winlogbeat security module's dashboard

New Dashboards

    User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections.
    Failed and Blocked Accounts allow us to keep track to failed logons and locked out account

Existing Dashboards

    Added Distribution groups Events (#15217)
    Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added

All Dashboards

    Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards)
    image

    Visualization that use may events (like group management related visualizations)
    were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization)

    Removed the margin between panels to look in the same way that other beats dashboards

    TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
…lastic#18775)

This PR adds two new dashboards related to events added in PRs (elastic#12906, elastic#14299, elastic#15217, elastic#17517) and implements some improvements to existing winlogbeat security module's dashboard

New Dashboards

    User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections.
    Failed and Blocked Accounts allow us to keep track to failed logons and locked out account

Existing Dashboards

    Added Distribution groups Events (elastic#15217)
    Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added

All Dashboards

    Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards)
    image

    Visualization that use may events (like group management related visualizations)
    were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization)

    Removed the margin between panels to look in the same way that other beats dashboards

    TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
andrewkroh pushed a commit to andrewkroh/beats that referenced this pull request Nov 16, 2020
…lastic#18775)

This PR adds two new dashboards related to events added in PRs (elastic#12906, elastic#14299, elastic#15217, elastic#17517) and implements some improvements to existing winlogbeat security module's dashboard

New Dashboards

    User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections.
    Failed and Blocked Accounts allow us to keep track to failed logons and locked out account

Existing Dashboards

    Added Distribution groups Events (elastic#15217)
    Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added

All Dashboards

    Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards)
    image

    Visualization that use may events (like group management related visualizations)
    were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization)

    Removed the margin between panels to look in the same way that other beats dashboards

    TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
(cherry picked from commit 7b9c535)
andrewkroh added a commit that referenced this pull request Nov 30, 2020
…18775) (#22598)

This PR adds two new dashboards related to events added in PRs (#12906, #14299, #15217, #17517) and implements some improvements to existing winlogbeat security module's dashboard

New Dashboards

    User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections.
    Failed and Blocked Accounts allow us to keep track to failed logons and locked out account

Existing Dashboards

    Added Distribution groups Events (#15217)
    Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added

All Dashboards

    Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards)
    image

    Visualization that use may events (like group management related visualizations)
    were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization)

    Removed the margin between panels to look in the same way that other beats dashboards

    TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
(cherry picked from commit 7b9c535)

Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants