Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup static mode after the otel upgrade, fix bug with Jaeger remote sampling #4640

Merged
merged 4 commits into from
Aug 1, 2023

Conversation

ptodev
Copy link
Contributor

@ptodev ptodev commented Jul 28, 2023

PR Description

There is an issue with Jaeger remote sampling not working after #3858 was merged. This PR will fix it.
I also updated the docs, and updated other components in the Static mode's tracing subsystem to use new parameters which were not previously available on the old Otel version.

Notes to the Reviewer

  • We will need to create a patch release for v0.35 once this PR is merged to main.
  • I wish I could add more tests - for example, for marshalling to "secret".
  • I wish there could be an error if there is an empty jaeger_remote_sampling: in the config. However, I could not get that to work.
  • This is not the last fix which has to go to 0.35.3. I will also need to fix another bug due to the Otel upgrade, but will do it on a separate PR.

PR Checklist

  • CHANGELOG updated
  • Documentation added
  • Tests updated

@ptodev ptodev requested review from a team and clayton-cornell as code owners July 28, 2023 17:29
@ptodev ptodev changed the title Cleanup static mode after the otel upgrade Cleanup static mode after the otel upgrade, fix bug with Jaeger remote sampling Jul 28, 2023
@ptodev ptodev force-pushed the static-mode-cleanup-after-otel-upgrade branch from 9d6ffd8 to 8299cdb Compare July 31, 2023 09:53
Comment on lines 24 to 28
### Breaking change: Jaeger remote sampling does not work

Jaeger remote sampling used to be configured via the jaeger receiver. This receiver was updated to a new version, and remote sampling has stopped working.

Jaeger remote sampling will be reintroduced in v35.3. It will be configured separately from the jaeger receiver.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if we should introduce new functionality in a patch release; it goes against semver. We could say this is a known issue in v0.35 and to wait until v0.36 for a fix.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is something that is currently broken then fixing it in a patch via new functionality feels reasonable. The other option that is most semver compatible is to roll back the otel upgrade. Given the effort there that is painful to do. @ptodev have we gotten any reports on this or did we find it?

I guess fundamentally though we could roll out a v0.36 instead of a patch if we wanted to align with semver?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm not going to push too hard on this, but it does feel bad to categorize it as a breaking change, since that feels more deliberate/planned, and this is just an artifact of upgrading.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, this feels like a breaking change that became a bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have we gotten any reports on this or did we find it?

It was reported internally by another team in Grafana.

it does feel bad to categorize it as a breaking change, since that feels more deliberate/planned, and this is just an artifact of upgrading

I agree. The reason I listed it as a breaking change is to raise awareness in case people are yet to upgrade, or have upgraded and experienced issues. I could clarify that this was an unintentional breaking change, a bug, and the bug will be fixed in v35.3? I don't think it goes against semver, since I really think of this as a bugfix rather than a new feature.

Copy link
Member

@rfratto rfratto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM following my feedback on the breaking change section.

docs/sources/flow/upgrade-guide.md Outdated Show resolved Hide resolved
Copy link
Contributor

@clayton-cornell clayton-cornell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No additional changes for docs. It's fine as-is

@ptodev ptodev force-pushed the static-mode-cleanup-after-otel-upgrade branch from 026d419 to 492a016 Compare August 1, 2023 16:47
@ptodev
Copy link
Contributor Author

ptodev commented Aug 1, 2023

Thank you for the reviews. I updated the doc with Robert's suggestions, added example configs to the upgrade guide, and also moved my changes from the flow upgrade guide to the static one (I didn't notice it earlier, apologies 🤦‍♂️ ). I'll merge this when the CI finishes.

@ptodev ptodev merged commit 8d82b8f into main Aug 1, 2023
6 checks passed
@ptodev ptodev deleted the static-mode-cleanup-after-otel-upgrade branch August 1, 2023 16:53
ptodev added a commit that referenced this pull request Aug 4, 2023
…e sampling (#4640)

* Cleanup static mode after the otel upgrade

* Fix linter issues

* Update docs/sources/flow/upgrade-guide.md

Co-authored-by: Robert Fratto <robertfratto@gmail.com>

* Move breaking change note to static mode doc, add example configs

---------

Co-authored-by: Robert Fratto <robertfratto@gmail.com>
clayton-cornell pushed a commit that referenced this pull request Aug 14, 2023
…e sampling (#4640)

* Cleanup static mode after the otel upgrade

* Fix linter issues

* Update docs/sources/flow/upgrade-guide.md

Co-authored-by: Robert Fratto <robertfratto@gmail.com>

* Move breaking change note to static mode doc, add example configs

---------

Co-authored-by: Robert Fratto <robertfratto@gmail.com>
clayton-cornell pushed a commit that referenced this pull request Aug 14, 2023
…e sampling (#4640)

* Cleanup static mode after the otel upgrade

* Fix linter issues

* Update docs/sources/flow/upgrade-guide.md

Co-authored-by: Robert Fratto <robertfratto@gmail.com>

* Move breaking change note to static mode doc, add example configs

---------

Co-authored-by: Robert Fratto <robertfratto@gmail.com>
@github-actions github-actions bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Feb 22, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants