Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat][broker] PIP-264: Enable OpenTelemetry reusable data memory mode #22914

Merged
merged 2 commits into from
Jun 16, 2024

Conversation

dragosvictor
Copy link
Contributor

@dragosvictor dragosvictor commented Jun 14, 2024

PIP: https://github.com/apache/pulsar/blob/master/pip/pip-264.md

Motivation

The recently added OpenTelemetry 'reusable data' memory mode allows the OpenTelemetry SDK to reduce the number of memory allocations in between collector runs, at the expense of parallel collection. This is a fair compromise given Pulsar's use-case, and it promises to significantly improve the performance of the metric pipeline.

Modifications

Enables, by default, the OpenTelemetry 'reusable data' export mode. This can be overridden by the user at runtime via Java system property otel.java.experimental.exporter.memory_mode or environment variable OTEL_JAVA_EXPERIMENTAL_EXPORTER_MEMORY_MODE. See https://opentelemetry.io/docs/languages/java/configuration/#exporters for a few more details on the config.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is already covered by existing tests, such as:

  • Integration test org.apache.pulsar.tests.integration.metrics.OpenTelemetrySanityTest

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations: As described above
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: dragosvictor#32

@dragosvictor dragosvictor marked this pull request as ready for review June 14, 2024 23:48
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jun 14, 2024
@heesung-sn heesung-sn requested a review from asafm June 15, 2024 01:16
@lhotari lhotari requested a review from merlimat June 15, 2024 08:48
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lhotari lhotari merged commit 1a6254a into apache:master Jun 16, 2024
50 of 53 checks passed
@dragosvictor dragosvictor deleted the dmisca-pip-264-otel-reusable-data branch June 17, 2024 17:09
@lhotari lhotari added this to the 4.0.0 milestone Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants