Backport PR #13391 to 8.0: ECS on by default for Logstash 8, again #13416
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* ecs: report pipeline's ECS-compatibility with INFO at startup Because the pipeline-level setting `pipeline.ecs_compatibility` affects the default behaviour of nearly every plugin in the pipeline, an INFO-level log message will provide useful hints, especially to our users who upgrade to Logstash 8 without first reading the breaking changes docs. For example, when we have two pipelines `old` and `new` whose `pipeline.ecs_compatibility` is `disabled` and `v8` respectively, we would get the following log messages: > ~~~ > [2021-11-04T18:43:21,810][INFO ][logstash.javapipeline ] Pipeline `old` is configured with `pipeline.ecs_compatibility: disabled` setting. All plugins in this pipeline will default to `ecs_compatibility => disabled` unless explicitly configured otherwise. > [2021-11-04T18:43:21,817][INFO ][logstash.javapipeline ] Pipeline `new` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise. > ~~~ * ecs: make v8 the default for 8.0 * ecs: `pipeline.ecs_compatibility` defaults to `v8` Related: elastic#11623 * doc: temporarily remove deep link from breaking changes doc to fix build (cherry picked from commit c3e498a)
kares
approved these changes
Nov 18, 2021
There was a problem hiding this comment.
LGTM, tested a few scenarios,
including ingestion to ES with defaults (output { elasticsearch {} }) - had the proper warning about ECS v8 N/A:
[WARN ][logstash.outputs.elasticsearch][main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport PR #13391 to 8.0 branch. Original message:
Release notes
[breaking] The
pipeline.ecs_compatibilitysetting now defaults tov8, causing all plugins that implement an ECS compatibility mode to default to operating withecs_compatibility => v8.What does this PR do?
Effectively reverts #13080, making the default ECS compatibility mode
v8for Logstash 8, and adds INFO-level logging for each pipeline to indicate in which mode its plugins will be run.Why is it important/What is the impact to the user?
This the final step of a large and ongoing effort to provide better support for the Elastic Common Schema, detailed in #11623. In #13080 we had back-tracked making the breaking change out of an abundance of caution, but have since decided to move forward with it.
Checklist
Author's Checklist
There are a number of plugins that ship with Logstash and have ECS modes, that do NOT yet have modes for v8. Because the scope of changes for v1->v8 is so small, I think that all of these can provide a v8 implementation that simply aliases the existing v1 implementation.
How to test this PR locally
bin/logstash -e ''[host][hostname])Related issues
Resolves #11623
Reverts #13080