Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance GPU Management, Operator Control, and Multiprocessing #354

Merged
merged 8 commits into from
Jul 17, 2024

Conversation

drcege
Copy link
Collaborator

@drcege drcege commented Jul 15, 2024

This PR enhances GPU resource management, individualized operator control, and robust multiprocessing support.

Optimize EasyOCR GPU Memory Usage

Fix Per-Operator Accelerator Support

  • Remove global use_cuda() checks and switch to per-operator checks within the executor loop.
  • Implement a per-operator op.use_cuda() method in the base OP class, considering both GPU availability and self.accelerator attribute.
  • Refactor accelerator parsing and configuration logic to ensure proper setup, and ensure compatibility with jsonargparse for argument registration.
  • Add missing accelerator configurations for CUDA-related operators. @HYLcool, please verify if any are still missing.

Implement Per-Operator Multiprocessing Method

  • Introduce a UNFORKABLE decorator to record operators incompatible with the fork mode.
  • Shift from a global multiprocessing strategy to per-operator runtime settings.
  • Update unit tests to automatically switch to spawn mode based on GPU availability.

- fix easyocr
- refine op-level cuda
- refine op-level mp
@drcege drcege added bug Something isn't working enhancement New feature or request labels Jul 15, 2024
@drcege drcege self-assigned this Jul 15, 2024
@drcege drcege changed the title [Testing] Improve model loading Improve model loading Jul 16, 2024
@drcege drcege changed the title Improve model loading Enhance GPU Management, Operator Control, and Multiprocessing Jul 17, 2024
@drcege drcege requested a review from BeachWang July 17, 2024 09:32
Copy link
Collaborator

@yxdyc yxdyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yxdyc yxdyc merged commit 5d4e08c into main Jul 17, 2024
3 checks passed
@yxdyc yxdyc mentioned this pull request Jul 17, 2024
12 tasks
@HYLcool HYLcool deleted the enhance/model_cuda_mp branch December 23, 2024 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

video_ocr_area_ratio_filter may cause cuda out of memory
3 participants