Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed #18979

vjsamuel · 2020-06-04T17:03:43Z

Type of PR

Enhancement

What does this PR do?

I have enhanced the emitEvent to generate a pod level event with no port. The hints builder checks to see if there is a port or not. If not then it uses the pod level metadata to monitor data.host:<port> annotations. This ensures that we dont use the wrong container metadata to start up a metricbeat module or do it multiple times as well.

Why is it important?

The way that things exist today is that when ports arent exposed, the container meta is non-deterministic

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

Fixes: #12011

elasticmachine · 2020-06-04T17:03:44Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

elasticmachine · 2020-06-04T17:03:44Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

elasticmachine · 2020-06-04T17:07:47Z

💚 Build Succeeded

Expand to view the summary

Build stats

Build Cause: [jsoriano commented: jenkins, run the tests please]
Start Time: 2020-06-16T15:25:10.040+0000
Duration: 79 min 8 sec

Test stats 🧪

Test	Results
Failed	0
Passed	9441
Skipped	1574
Total	11015

Steps errors

Expand to view the steps failures

Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Duration: 2 min 22 sec
- Start Time: 2020-06-16T16:01:05.557+0000
- log
Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Duration: 2 min 22 sec
- Start Time: 2020-06-16T16:03:27.465+0000
- log

elasticmachine · 2020-06-05T08:12:00Z

Pinging @elastic/integrations-platforms (Team:Platforms)

jsoriano

Hey @vjsamuel, thanks for this fix.

As you mention, this PR includes two things, one fix to avoid mapping incorrect ports, and an improvement on how beats generate autodiscover events for pods. Could you please separate the change in two PRs?
I think that the enhancement may require further discussion and I wouldn't like to block one thing with the other.

Changelogs will be needed.

libbeat/autodiscover/providers/kubernetes/pod.go

metricbeat/autodiscover/builder/hints/metrics.go

jsoriano · 2020-06-12T15:44:21Z

I think we can go on with this change. @vjsamuel could you take a look to the merge conflict?

jsoriano · 2020-06-12T15:55:03Z

jenkins, run the tests please

jsoriano · 2020-06-15T18:12:50Z

jenkins, run the tests please

…n port is not exposed

jsoriano · 2020-06-16T15:24:38Z

jenkins, run the tests please

…n port is not exposed (elastic#18979) (cherry picked from commit 2f7b501)

…n port is not exposed (#18979) (#19233) (cherry picked from commit 2f7b501) Co-authored-by: Vijay Samuel <vjsamuel@ebay.com>

#18979 introduced a pod level event which is generated after all container events. The ordering is wrong in that pod events are sent last which would generate a valid event similar to container events. The ordering needs to be pod first and container events next so that pod events dont override valid container events. One other issue was that the pod level hint generates a single config with all hosts and it wont get over written by container hints causing more than one config to be spun up for the same hint (one with a container meta and one without).

elastic#18979 introduced a pod level event which is generated after all container events. The ordering is wrong in that pod events are sent last which would generate a valid event similar to container events. The ordering needs to be pod first and container events next so that pod events dont override valid container events. One other issue was that the pod level hint generates a single config with all hosts and it wont get over written by container hints causing more than one config to be spun up for the same hint (one with a container meta and one without). (cherry picked from commit 58edbb4)

#18979 introduced a pod level event which is generated after all container events. The ordering is wrong in that pod events are sent last which would generate a valid event similar to container events. The ordering needs to be pod first and container events next so that pod events dont override valid container events. One other issue was that the pod level hint generates a single config with all hosts and it wont get over written by container hints causing more than one config to be spun up for the same hint (one with a container meta and one without). (cherry picked from commit 58edbb4) Co-authored-by: Vijay Samuel <vjsamuel@ebay.com>

…n port is not exposed (elastic#18979)

elastic#18979 introduced a pod level event which is generated after all container events. The ordering is wrong in that pod events are sent last which would generate a valid event similar to container events. The ordering needs to be pod first and container events next so that pod events dont override valid container events. One other issue was that the pod level hint generates a single config with all hosts and it wont get over written by container hints causing more than one config to be spun up for the same hint (one with a container meta and one without).

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 4, 2020

vjsamuel force-pushed the fix_port_overlaps branch from 1d6c0c1 to 7083347 Compare June 4, 2020 17:34

ChrsMark added [zube]: In Review review Team:Platforms Label for the Integrations - Platforms team labels Jun 5, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 5, 2020

zube bot added [zube]: Inbox [zube]: In Review and removed [zube]: In Review [zube]: Inbox labels Jun 5, 2020

jsoriano reviewed Jun 8, 2020

View reviewed changes

libbeat/autodiscover/providers/kubernetes/pod.go Show resolved Hide resolved

libbeat/autodiscover/providers/kubernetes/pod.go Show resolved Hide resolved

metricbeat/autodiscover/builder/hints/metrics.go Outdated Show resolved Hide resolved

vjsamuel force-pushed the fix_port_overlaps branch 5 times, most recently from bd11535 to 5d1e81c Compare June 10, 2020 20:31

vjsamuel force-pushed the fix_port_overlaps branch from 5d1e81c to e0a6431 Compare June 12, 2020 15:46

vjsamuel force-pushed the fix_port_overlaps branch 2 times, most recently from ee02af1 to 08baa61 Compare June 15, 2020 06:01

Fix metrics hints builder to avoid wrong container metadata usage whe…

1a5eac8

…n port is not exposed

vjsamuel force-pushed the fix_port_overlaps branch from 08baa61 to 1a5eac8 Compare June 15, 2020 23:30

jsoriano self-assigned this Jun 16, 2020

jsoriano approved these changes Jun 16, 2020

View reviewed changes

jsoriano merged commit 2f7b501 into elastic:master Jun 16, 2020

zube bot added [zube]: Done and removed [zube]: In Review labels Jun 16, 2020

jsoriano mentioned this pull request Jun 17, 2020

Cherry-pick #18979 to 7.x: Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed #19233

Merged

6 tasks

jsoriano added the v7.9.0 label Jun 17, 2020

jsoriano pushed a commit to jsoriano/beats that referenced this pull request Jun 17, 2020

Fix metrics hints builder to avoid wrong container metadata usage whe…

3e7b675

…n port is not exposed (elastic#18979) (cherry picked from commit 2f7b501)

jsoriano added test-plan Add this PR to be manual test plan and removed test-plan Add this PR to be manual test plan labels Jun 17, 2020

jsoriano added a commit that referenced this pull request Jun 18, 2020

Fix metrics hints builder to avoid wrong container metadata usage whe…

f65b237

…n port is not exposed (#18979) (#19233) (cherry picked from commit 2f7b501) Co-authored-by: Vijay Samuel <vjsamuel@ebay.com>

vjsamuel deleted the fix_port_overlaps branch June 22, 2020 06:14

vjsamuel mentioned this pull request Jun 22, 2020

Fix ordering and duplicate configs on autodiscover #19317

Merged

6 tasks

andresrc removed the [zube]: Done label Jun 22, 2020

ChrsMark mentioned this pull request Jun 25, 2020

Kubernetes autodiscover - multiple ports #15796

Closed

jsoriano mentioned this pull request Jul 2, 2020

Cherry-pick #19317 to 7.x: Fix ordering and duplicate configs on autodiscover #19614

Merged

6 tasks

jsoriano mentioned this pull request Aug 21, 2020

[autodiscover] Error creating runner from config: Can only start an input when all related states are finished #11834

Closed

melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020

Fix metrics hints builder to avoid wrong container metadata usage whe…

603d344

…n port is not exposed (elastic#18979)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed #18979

Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed #18979

vjsamuel commented Jun 4, 2020 •

edited

Loading

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Jun 5, 2020

jsoriano left a comment

jsoriano commented Jun 12, 2020

jsoriano commented Jun 12, 2020

jsoriano commented Jun 15, 2020

jsoriano commented Jun 16, 2020

Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed #18979

Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed #18979

Conversation

vjsamuel commented Jun 4, 2020 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Steps errors

elasticmachine commented Jun 5, 2020

jsoriano left a comment

Choose a reason for hiding this comment

jsoriano commented Jun 12, 2020

jsoriano commented Jun 12, 2020

jsoriano commented Jun 15, 2020

jsoriano commented Jun 16, 2020

vjsamuel commented Jun 4, 2020 •

edited

Loading

elasticmachine commented Jun 4, 2020 •

edited

Loading