Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set persistent_workers and pin_memory as True #2224

Merged

Conversation

eunwoosh
Copy link
Contributor

@eunwoosh eunwoosh commented Jun 7, 2023

Summary

Set persistent_workers and pin_memory as True in all tasks, which can decrease data loader time.
When training with persistent_workers in nncf, nncf saves dataloaders in cfg. It makes an error when deepcopy cfg in train_model function of each tasks because datalaoder with persistent_workers aren't pickleable.
So, I set persistent_workers as False in nncf.

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@eunwoosh eunwoosh added this to the 1.4.0 milestone Jun 7, 2023
@github-actions github-actions bot added ALGO Any changes in OTX Algo Tasks implementation DOC Improvements or additions to documentation labels Jun 7, 2023
@eunwoosh eunwoosh marked this pull request as ready for review June 7, 2023 08:16
@eunwoosh eunwoosh requested a review from a team as a code owner June 7, 2023 08:16
sungmanc
sungmanc previously approved these changes Jun 8, 2023
@eunwoosh eunwoosh changed the title Set persistent_workers and pin_memory as True in detection task Set persistent_workers and pin_memory as True Jun 8, 2023
@eunwoosh eunwoosh force-pushed the es/det_persistent_worker_true branch from 1f9e4d4 to 02be8d7 Compare June 8, 2023 04:46
@eunwoosh eunwoosh enabled auto-merge (squash) June 8, 2023 07:54
@eunwoosh eunwoosh requested a review from sungmanc June 8, 2023 07:55
@eunwoosh eunwoosh merged commit 0033a46 into openvinotoolkit:develop Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ALGO Any changes in OTX Algo Tasks implementation DOC Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants