Improve forking support #804
Merged
+276
−46
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of separate locks for buffering and sender modes configuration, use the same lock.
Now that disable_buffering and pre_fork may both try to stop the flush thread at the same time, attempt to stop an already stopped thread does not indicate a bug in the client code.
Single field assignement is atomic according to Python FAQ, so we don't need to protect reads and writes to self._queue as such. The only place where we need a lock is when stopping the sender, to ensure that the Stop marker is the last thing to be placed in there. https://docs.python.org/3/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe
os.register_at_fork is not available on all python versions, notably python 2.7. While we check for the availability explicitly, mypy cannot see through that and complains.
vickenty
added
the
changelog/Added
The method is now thread safe, and can be used with forking applications.
Re-format parameter descriptions so they are rendered correctly in the html docs.
gh123man
previously approved these changes
Jan 9, 2024
skarimo
previously approved these changes
Jan 10, 2024
gh123man
approved these changes
Jan 10, 2024
skarimo
approved these changes
Jan 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Use
register_at_forkadded in Python 3.7+ to transparently handle process fork when the client is using background threads.Description of the Change
Forking support is introduced in several layers:
os.register_at_fork.This approach is similar to one used in dd-trace-py for handling forks.
Alternate Designs
Possible Drawbacks
Verification Process
Additional Notes
Release Notes
Review checklist (to be filled by reviewers)
changelog/label attached. If applicable it should have thebackward-incompatiblelabel attached.do-not-merge/label attached.kind/andseverity/labels attached at least.