This repository has been archived by the owner on Sep 18, 2024. It is now read-only.
Fix on log and metric issue on Windows when using pytorch with multithread #3604
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The problem with log and metric when using torch.utils.data.DataLoader with num_workers>0 on windows is known for a long time. I find that the log and metric file is whipped of its content whenever a DataLoader is enumerated. I reached out to pytorch for this, and they told me it is a issue with the multiprocessing library. It uses on default spawn mode in windows, in which
This cause the log and metric file which are opened at top level in
__init__.py
andruntime/platform/local.py
. Thankfully there is a simple workaround on this, that is to change the file to 'a' open mode. I tested it on my local machine and it works.