Disable migration aliases in index pattern #10478
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
houndci-bot
reviewed
Feb 1, 2019
This was referenced Feb 1, 2019
ruflin
commented
Feb 1, 2019
ruflin
force-pushed
the
index-pattern-generation-cleanup
branch
from
5e1f90a
to
6bebce2
Compare
ruflin
changed the title
ruflin
added
review
and removed
in progress
ruflin
force-pushed
the
index-pattern-generation-cleanup
branch
from
6bebce2
to
5b4c346
Compare
ruflin
changed the title
ruflin
force-pushed
the
index-pattern-generation-cleanup
branch
from
5b4c346
to
856674c
Compare
ruflin
commented
Feb 4, 2019
| @@ -21,6 +21,7 @@ The list below covers the major changes between 7.0.0-alpha2 and master only. | |||
| ==== Breaking changes | |||
| - Outputs receive Index Manager as additional parameter. The index manager can | |||
| be used to create an index selector. {pull}10347[10347] | |||
| - Remove support for loading dashboards to Elasticsearch 5. {pull}10451[10451] | |||
There was a problem hiding this comment.
This was missing in a previous PR.
simitt
reviewed
Feb 4, 2019
The migration aliases should not show up in the index pattern if `migration.enabled: false`. For this to happen, the Kibana index pattern must be generated on the fly instead of packaging it with each Beat. This PR introduces the generation of the index pattern when Kibana data is loaded. APM still needs the index pattern as file. For this the export command `index-pattern` was added. It will print the index pattern to the standard out: ``` ./metricbeat export index-pattern > pattern.json ``` The commands to generate the index pattern in the dev environment were removed. For checking if aliases are supported, the Kibana version is checked. Fully accurate would be to check the Elasticsearch version as it depends on the ES version in the end and not Kibana. But it's assume that in general the same minor version is used. The reason not Elasticsearch is checked as it would potentially require additional config options and adds unnecessary complexity. For the index pattern the internal fields.go are used. Even if fields.yml is configured still fields.go is used. This is the same behavior as we had so far when the index pattern was generated. It could be improved in the future to also support a fields.yml for the generation if needed. In general this PR tried to change as little code as possible. The code and tests around the Kibana dashboard generation and index pattern generation is not very nice. One reason is that it also contains old logic which was used for previous versions but also has the ability to read dashboards from a zip file. Because of this all the old capabilities have to stay in the code for now. The code should be cleaned up at a later stage. Further changes: * Added system tests to Filebeat to check for correct content in index pattern when migration is enabled. * Fix double generation of common.yml in Metricbeat. It seems some changes in the past caused that the common.yml file was contained in two fields.go files in Metricbeat
ruflin
force-pushed
the
index-pattern-generation-cleanup
branch
from
253c0a1
to
c3341db
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The migration aliases should not show up in the index pattern if
migration.enabled: false. For this to happen, the Kibana index pattern must be generated on the fly instead of packaging it with each Beat. This PR introduces the generation of the index pattern when Kibana data is loaded.APM still needs the index pattern as file. For this the export command
index-patternwas added. It will print the index pattern to the standard out:The commands to generate the index pattern in the dev environment were removed.
For checking if aliases are supported, the Kibana version is checked. Fully accurate would be to check the Elasticsearch version as it depends on the ES version in the end and not Kibana. But it's assume that in general the same minor version is used. The reason not Elasticsearch is checked as it would potentially require additional config options and adds unnecessary complexity.
For the index pattern the internal fields.go are used. Even if fields.yml is configured still fields.go is used. This is the same behavior as we had so far when the index pattern was generated. It could be improved in the future to also support a fields.yml for the generation if needed.
In general this PR tried to change as little code as possible. The code and tests around the Kibana dashboard generation and index pattern generation is not very nice. One reason is that it also contains old logic which was used for previous versions but also has the ability to read dashboards from a zip file. Because of this all the old capabilities have to stay in the code for now. The code should be cleaned up at a later stage.
Further changes: