Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate docker autodiscovery to ECS #10898

Merged
merged 16 commits into from
Feb 25, 2019

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Feb 22, 2019

This PR includes #10862 and #10758

As a summary:

  • Autodiscover selectors using ECS structure are added to autodiscover events, old selectors are kept for backwards compatibility
  • Autodiscover generated metadata follows ECS
  • Dedotting of labels is added, enabled by default, will be backported for 6.7, but disabled

docker.containers.labels is not migrated, as it wasn't for add_docker_metadata (see #9412 (comment))

Fixes #10757

Co-Authored-By: kaiyan-sheng kaiyan.sheng@elastic.co
Co-Authored-By: Nicolas Ruflin spam@ruflin.com

@jsoriano jsoriano added Filebeat Filebeat libbeat containers Related to containers use case v7.0.0 ecs Team:Integrations Label for the Integrations team v6.7.0 labels Feb 22, 2019
@jsoriano jsoriano self-assigned this Feb 22, 2019
@jsoriano jsoriano requested review from a team as code owners February 22, 2019 16:53
@jsoriano jsoriano requested review from a team and removed request for a team February 22, 2019 17:41
@kaiyan-sheng
Copy link
Contributor

@jsoriano Thanks for fixing the selector part! This is tested and works! But the CI failed because

======================================================================
ERROR: jolokia autodiscover with jmx metricset
Exception: Key 'jolokia.agent.version' found in event is not documented!

So we just need to document it then :-)

@jsoriano
Copy link
Member Author

@kaiyan-sheng I have removed the check for jolokia from this PR, lets add these fields in a follow up, at the moment we don't have fields files for autodiscover providers.

@kaiyan-sheng
Copy link
Contributor

sounds good to me! Thanks!

@kaiyan-sheng
Copy link
Contributor

jenkins, test this please

@jsoriano jsoriano merged commit 1bf8087 into elastic:master Feb 25, 2019
@jsoriano jsoriano deleted the autodiscovery-ecs-docker-7.0 branch February 25, 2019 09:30
@jsoriano
Copy link
Member Author

@kaiyan-sheng issue created for Jolokia fields #10925

jsoriano added a commit to jsoriano/beats that referenced this pull request Feb 25, 2019
Fields injected by docker autodiscover provider were being placed
in alias fields introduced for ECS, change them to the new location
and add selectors accordingly. 

This PR includes elastic#10862 and elastic#10758

As a summary:
    * Autodiscover selectors using ECS structure are added to
      autodiscover events, old selectors are kept for backwards compatibility
    * Autodiscover generated metadata follows ECS
    * Dedotting of labels is added, enabled by default, will be backported for 6.7,
      but disabled

`docker.containers.labels` is not migrated, as it wasn't for `add_docker_metadata`
(see elastic#9412)

Fixes elastic#10757

Co-Authored-By: kaiyan-sheng <kaiyan.sheng@elastic.co>
Co-Authored-By: Nicolas Ruflin <spam@ruflin.com>
(cherry picked from commit 1bf8087)
jsoriano added a commit that referenced this pull request Feb 26, 2019
Partial backport of #10898 to 6.7

Summary of backported changes:
    * ECS fields are added as autodiscover selectors
    * Labels dedotting added, disabled by default

Co-Authored-By: kaiyan-sheng <kaiyan.sheng@elastic.co>
Co-Authored-By: Nicolas Ruflin <spam@ruflin.com>
jsoriano added a commit that referenced this pull request Feb 27, 2019
Fields injected by docker autodiscover provider were being placed
in alias fields introduced for ECS, change them to the new location
and add selectors accordingly. 

This PR includes #10862 and #10758

As a summary:
    * Autodiscover selectors using ECS structure are added to
      autodiscover events, old selectors are kept for backwards compatibility
    * Autodiscover generated metadata follows ECS
    * Dedotting of labels is added, enabled by default, will be backported for 6.7,
      but disabled

`docker.containers.labels` is not migrated, as it wasn't for `add_docker_metadata`
(see #9412)

Fixes #10757

(cherry picked from commit 1bf8087)

Co-Authored-By: kaiyan-sheng <kaiyan.sheng@elastic.co>
Co-Authored-By: Nicolas Ruflin <spam@ruflin.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug containers Related to containers use case ecs Filebeat Filebeat libbeat review Team:Integrations Label for the Integrations team v6.7.0 v7.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants