Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat][New Module] Add support for Microsoft MTP / 365 Defender #21446

Merged
merged 22 commits into from
Oct 6, 2020
Merged

[Filebeat][New Module] Add support for Microsoft MTP / 365 Defender #21446

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
64f53dc
Initial commit for mtp mvp
P1llus Sep 24, 2020
ec17e1a
first finished MVP version of MTP module
P1llus Sep 28, 2020
f495bd3
updating m365_defender with new fields and new name
P1llus Oct 1, 2020
b772e44
reverting some files that shouldnt be added
P1llus Oct 1, 2020
c7d7a67
Merge branch 'master' into filebeat_mtp_mvp
P1llus Oct 1, 2020
a79db22
Merge branch 'master' into filebeat_mtp_mvp
P1llus Oct 2, 2020
218996a
removing dhcp generated logs from PR
P1llus Oct 5, 2020
d928524
converting two fields to strings and updating some default template c…
P1llus Oct 5, 2020
ed695e2
adding changelog entry
P1llus Oct 5, 2020
6efcb68
Merge branch 'master' into filebeat_mtp_mvp
P1llus Oct 5, 2020
41cd630
Initial commit for mtp mvp
P1llus Sep 24, 2020
9b08557
first finished MVP version of MTP module
P1llus Sep 28, 2020
eb54d0b
updating m365_defender with new fields and new name
P1llus Oct 1, 2020
be43949
reverting some files that shouldnt be added
P1llus Oct 1, 2020
b6569cd
removing dhcp generated logs from PR
P1llus Oct 5, 2020
f996382
converting two fields to strings and updating some default template c…
P1llus Oct 5, 2020
2beebc7
adding changelog entry
P1llus Oct 5, 2020
8845fdc
Merge branch 'filebeat_mtp_mvp' of github.com:P1llus/beats into fileb…
P1llus Oct 5, 2020
7a4eccb
Merge branch 'master' into filebeat_mtp_mvp
P1llus Oct 5, 2020
96ee68b
updating typo
P1llus Oct 6, 2020
abbd0cd
Merge branch 'master' into filebeat_mtp_mvp
marc-gr Oct 6, 2020
b494e21
Merge branch 'master' into filebeat_mtp_mvp
marc-gr Oct 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
427 changes: 427 additions & 0 deletions filebeat/docs/fields.asciidoc
View file
Open in desktop

Large diffs are not rendered by default.

84 changes: 82 additions & 2 deletions filebeat/docs/modules/microsoft.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ This file is generated! See scripts/docs_collector.py

This is a module for ingesting data from the different Microsoft Products. Currently supports these filesets:

- `defender_atp` fileset: Supports Microsoft Defender ATP
- `defender_atp` fileset: Supports Microsoft Defender for Endpoint (Microsoft Defender ATP)
- `m365_defender` fileset: Supports Microsoft 365 Defender (Microsoft Threat Protection)
- `dhcp` fileset: Supports Microsoft DHCP logs

include::../include/what-happens.asciidoc[]
Expand All @@ -25,6 +26,85 @@ include::../include/configuring-intro.asciidoc[]

include::../include/config-option-intro.asciidoc[]

[float]
==== `m365_defender` fileset settings

beta[]

To configure access for filebeat to Microsoft 365 Defender you will have to create a new Azure Application registration, this will again return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To configure access for filebeat to Microsoft 365 Defender you will have to create a new Azure Application registration, this will again return
To configure access for Filebeat to Microsoft 365 Defender you will have to create a new Azure Application registration, this will again return.

Oauth tokens with access to the Microsoft 365 Defender API

The procedure to create an application is found on the below link:

https://docs.microsoft.com/en-us/microsoft-365/security/mtp/api-create-app-web?view=o365-worldwide#create-an-app[Create a new Azure Application]

When giving the application the API permissions described in the documentation (Incident.Read.All) it will only grant access to read Incidents from 365 Defender and nothing else in the Azure Domain.

After the application has been created, it should contain 3 values that you need to apply to the module configuration.

These values are:

- Client ID
- Client Secret
- Tenant ID

Example config:

[source,yaml]
----
- module: microsoft
m365_defender:
enabled: true
var.oauth2.client.id: "123abc-879546asd-349587-ad64508"
var.oauth2.client.secret: "980453~-Sg99gedf"
var.oauth2.token_url: "https://login.microsoftonline.com/INSERT-TENANT-ID/oauth2/token"
----

*`var.oauth2.client.id`*::

This is the client ID related to creating a new application on Azure.

*`var.oauth2.client.secret`*::

The secret related to the client ID.

*`var.oauth2.token_url`*::

A predefined URL towards the Oauth2 service for Microsoft. The URL should always be the same with the exception of the Tenant ID that needs to be added to the full URL.

[float]
==== 365 Defender ECS fields

This is a list of 365 Defender fields that are mapped to ECS.

[options="header"]
|======================================================================
| 365 Defender Fields | ECS Fields |
| lastUpdateTime | @timestamp |
| severity | event.severity |
| createdTime | event.created |
| alerts.category | threat.technique.name |
| alerts.description | rule.description |
| alerts.serviceSource | event.provider |
| alerts.alertId | event.id |
| alerts.firstActivity | event.start |
| alerts.lastActivity | event.end |
| alerts.title | message |
| entities.processId | process.pid |
| entities.processCommandLine | process.command_line |
| entities.processCreationTime | process.start |
| entities.parentProcessId | process.parent.pid |
| entities.parentProcessCreationTime | process.parent.start |
| entities.sha1 | file.hash.sha1 |
| entities.sha256 | file.hash.sha256 |
| entities.url | url.full |
| entities.filePath | file.path |
| entities.fileName | file.name |
| entities.userPrincipalName | host.user.name |
| entities.domainName | host.user.domain |
| entities.aadUserId | host.user.id |
|======================================================================

[float]
==== `defender_atp` fileset settings

Expand Down Expand Up @@ -114,7 +194,7 @@ This module comes with a sample dashboard for Defender ATP.
[role="screenshot"]
image::./images/filebeat-defender-atp-overview.png[]

The best way to view Defender ATP events and alert data is in the SIEM.
The best way to view Defender ATP events and alert data is in the SIEM.

[role="screenshot"]
image::./images/siem-alerts-cs.jpg[]
Expand Down
15 changes: 14 additions & 1 deletion x-pack/filebeat/filebeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1105,7 +1105,20 @@ filebeat.modules:

# Oauth Client Secret
#var.oauth2.client.secret: ""


# Oauth Token URL, should include the tenant ID
#var.oauth2.token_url: "https://login.microsoftonline.com/TENANT-ID/oauth2/token"
m365_defender:
enabled: true
# How often the API should be polled
#var.interval: 5m

# Oauth Client ID
#var.oauth2.client.id: ""

# Oauth Client Secret
#var.oauth2.client.secret: ""

# Oauth Token URL, should include the tenant ID
#var.oauth2.token_url: "https://login.microsoftonline.com/TENANT-ID/oauth2/token"
dhcp:
Expand Down
15 changes: 14 additions & 1 deletion x-pack/filebeat/module/microsoft/_meta/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,20 @@

# Oauth Client Secret
#var.oauth2.client.secret: ""


# Oauth Token URL, should include the tenant ID
#var.oauth2.token_url: "https://login.microsoftonline.com/TENANT-ID/oauth2/token"
m365_defender:
enabled: true
# How often the API should be polled
#var.interval: 5m

# Oauth Client ID
#var.oauth2.client.id: ""

# Oauth Client Secret
#var.oauth2.client.secret: ""

# Oauth Token URL, should include the tenant ID
#var.oauth2.token_url: "https://login.microsoftonline.com/TENANT-ID/oauth2/token"
dhcp:
Expand Down
84 changes: 82 additions & 2 deletions x-pack/filebeat/module/microsoft/_meta/docs.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@

This is a module for ingesting data from the different Microsoft Products. Currently supports these filesets:

- `defender_atp` fileset: Supports Microsoft Defender ATP
- `defender_atp` fileset: Supports Microsoft Defender for Endpoint (Microsoft Defender ATP)
- `m365_defender` fileset: Supports Microsoft 365 Defender (Microsoft Threat Protection)
- `dhcp` fileset: Supports Microsoft DHCP logs

include::../include/what-happens.asciidoc[]
Expand All @@ -20,6 +21,85 @@ include::../include/configuring-intro.asciidoc[]

include::../include/config-option-intro.asciidoc[]

[float]
==== `m365_defender` fileset settings

beta[]

To configure access for filebeat to Microsoft 365 Defender you will have to create a new Azure Application registration, this will again return
Oauth tokens with access to the Microsoft 365 Defender API

The procedure to create an application is found on the below link:

https://docs.microsoft.com/en-us/microsoft-365/security/mtp/api-create-app-web?view=o365-worldwide#create-an-app[Create a new Azure Application]

When giving the application the API permissions described in the documentation (Incident.Read.All) it will only grant access to read Incidents from 365 Defender and nothing else in the Azure Domain.

After the application has been created, it should contain 3 values that you need to apply to the module configuration.

These values are:

- Client ID
- Client Secret
- Tenant ID

Example config:

[source,yaml]
----
- module: microsoft
m365_defender:
enabled: true
var.oauth2.client.id: "123abc-879546asd-349587-ad64508"
var.oauth2.client.secret: "980453~-Sg99gedf"
var.oauth2.token_url: "https://login.microsoftonline.com/INSERT-TENANT-ID/oauth2/token"
----

*`var.oauth2.client.id`*::

This is the client ID related to creating a new application on Azure.

*`var.oauth2.client.secret`*::

The secret related to the client ID.

*`var.oauth2.token_url`*::

A predefined URL towards the Oauth2 service for Microsoft. The URL should always be the same with the exception of the Tenant ID that needs to be added to the full URL.

[float]
==== 365 Defender ECS fields

This is a list of 365 Defender fields that are mapped to ECS.

[options="header"]
|======================================================================
| 365 Defender Fields | ECS Fields |
| lastUpdateTime | @timestamp |
| severity | event.severity |
| createdTime | event.created |
| alerts.category | threat.technique.name |
| alerts.description | rule.description |
| alerts.serviceSource | event.provider |
| alerts.alertId | event.id |
| alerts.firstActivity | event.start |
| alerts.lastActivity | event.end |
| alerts.title | message |
| entities.processId | process.pid |
| entities.processCommandLine | process.command_line |
| entities.processCreationTime | process.start |
| entities.parentProcessId | process.parent.pid |
| entities.parentProcessCreationTime | process.parent.start |
| entities.sha1 | file.hash.sha1 |
| entities.sha256 | file.hash.sha256 |
| entities.url | url.full |
| entities.filePath | file.path |
| entities.fileName | file.name |
| entities.userPrincipalName | host.user.name |
| entities.domainName | host.user.domain |
| entities.aadUserId | host.user.id |
|======================================================================

[float]
==== `defender_atp` fileset settings

Expand Down Expand Up @@ -109,7 +189,7 @@ This module comes with a sample dashboard for Defender ATP.
[role="screenshot"]
image::./images/filebeat-defender-atp-overview.png[]

The best way to view Defender ATP events and alert data is in the SIEM.
The best way to view Defender ATP events and alert data is in the SIEM.

[role="screenshot"]
image::./images/siem-alerts-cs.jpg[]
Expand Down
Loading