Skip to content

Commit

Permalink
Convert Filebeat mongodb.log to ECS (#10009)
Browse files Browse the repository at this point in the history
- Convert fields under `mongodb.log.*` to ECS. Previous field names are field aliases towards the new corresponding ECS field:
  - mongodb.log.message => message
  - mongodb.log.severity => log.level
  - read_timestamp => event.created (not aliased, still used elsewhere)
  • Loading branch information
webmat authored Jan 11, 2019
1 parent 27f7b15 commit f384ba3
Show file tree
Hide file tree
Showing 6 changed files with 134 additions and 133 deletions.
10 changes: 10 additions & 0 deletions dev-tools/ecs-migration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,16 @@
to: event.duration
alias: true

## MongoDB module

- from: mongodb.log.severity
to: log.level
alias: true

- from: mongodb.log.message
to: message
alias: true

## NGINX module

- from: nginx.access.user_name
Expand Down
26 changes: 11 additions & 15 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6265,18 +6265,6 @@ Contains fields from MongoDB logs.
*`mongodb.log.severity`*::
+
--
type: keyword
example: I
Severity level of message
--
*`mongodb.log.component`*::
+
--
Expand All @@ -6301,13 +6289,21 @@ Context of message
--
*`mongodb.log.message`*::
*`mongodb.log.severity`*::
+
--
type: text
type: alias
alias to: log.level
The message in the log line.
--
*`mongodb.log.message`*::
+
--
type: alias
alias to: message
--
Expand Down
2 changes: 1 addition & 1 deletion filebeat/module/mongodb/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 8 additions & 8 deletions filebeat/module/mongodb/log/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,6 @@
description: >
Contains fields from MongoDB logs.
fields:
- name: severity
description: >
Severity level of message
example: I
type: keyword
- name: component
description: >
Functional categorization of message
Expand All @@ -18,7 +13,12 @@
Context of message
example: initandlisten
type: keyword

- name: severity
type: alias
path: log.level
migration: true
- name: message
description: >
The message in the log line.
type: text
type: alias
path: message
migration: true
9 changes: 2 additions & 7 deletions filebeat/module/mongodb/log/ingest/pipeline.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,15 @@
"grok": {
"field": "message",
"patterns":[
"%{TIMESTAMP_ISO8601:mongodb.log.timestamp} %{WORD:mongodb.log.severity} %{WORD:mongodb.log.component} \\s*\\[%{WORD:mongodb.log.context}\\] %{GREEDYDATA:mongodb.log.message}"
"%{TIMESTAMP_ISO8601:mongodb.log.timestamp} %{WORD:log.level} %{WORD:mongodb.log.component} \\s*\\[%{WORD:mongodb.log.context}\\] %{GREEDYDATA:message}"
],
"ignore_missing": true
}
},
{
"remove": {
"field": "message"
}
},
{
"rename": {
"field": "@timestamp",
"target_field": "read_timestamp"
"target_field": "event.created"
}
},
{
Expand Down
Loading

0 comments on commit f384ba3

Please sign in to comment.