Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #20989 to 7.x: Disable Kafka metricsets based on Jolokia #21036

Merged
merged 1 commit into from
Sep 15, 2020

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Sep 9, 2020

Cherry-pick of PR #20989 to 7.x branch. Original message:

What does this PR do?

Kafka metricsets based on Jolokia require a different configuration to
the native metricsets. Disable the Jolokia ones by default, if someone
wants to use them, they need to explicitly enable and configure them.
Reference configuration contains information about this.

At the moment there is no way we can provide default metricsets in the
same module that need different hosts configurations.

In any case consumer and producer metricsets are intended to monitor
Java consumers and producers, while the rest of metricsets are intended
to monitor Kafka brokers. So there is little chance that all of them are going
to be enabled in the same configuration.

Why is it important?

To avoid errors with default configuration.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Use cases

Using the kafka module, without specifying defaults, should provide a working configuration.

Logs

Errors like these ones appear when using default configuration:

Error fetching data for metricset kafka.broker: error making http request: Post "http://pipeline-kafka:9092/jolokia/%3FignoreErrors=true&canonicalNaming=false": read tcp 10.5.7.136:34466->172.20.79.178:9092: read: connection reset by peer
Error fetching data for metricset kafka.producer: error making http request: Post "http://pipeline-kafka:9092/jolokia/%3FignoreErrors=true&canonicalNaming=false": read tcp 10.5.7.136:34470->172.20.79.178:9092: read: connection reset by peer

Metricbeat tries to request jolokia metrics from the Kafka endpoint.

Kafka metricsets based on Jolokia require a different configuration to
the native metricsets. Disable the Jolokia ones by default,  if someone
wants to use them, they need to explicitly enable and configure them.
Reference configuration contains information about this.

(cherry picked from commit cce29d4)
@jsoriano jsoriano added [zube]: In Review backport Team:Services (Deprecated) Label for the former Integrations-Services team labels Sep 9, 2020
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 9, 2020
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21036 opened]

  • Start Time: 2020-09-09T11:35:16.712+0000

  • Duration: 77 min 6 sec

Test stats 🧪

Test Results
Failed 0
Passed 3191
Skipped 691
Total 3882

@jsoriano jsoriano merged commit d75e6a9 into elastic:7.x Sep 15, 2020
@jsoriano jsoriano deleted the backport_20989_7.x branch September 15, 2020 09:02
@zube zube bot removed the [zube]: Done label Dec 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants