Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce _idle_semaphore calls in ThreadPoolExecutor #102024

Closed
jackcvr opened this issue Feb 18, 2023 · 2 comments
Closed

Reduce _idle_semaphore calls in ThreadPoolExecutor #102024

jackcvr opened this issue Feb 18, 2023 · 2 comments
Labels
performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@jackcvr
Copy link
Contributor

jackcvr commented Feb 18, 2023 •
edited by bedevere-bot
Loading

Feature or enhancement

In concurrent.futures.thread.ThredPoolExecutor method executor._idle_semaphore.release() should be called if only work queue is empty

Pitch

Currently _idle_semaphore.release() is called after processing every item in queue. That produces useless semaphore updates in case when queue still has items to process.
The optimization is going to increase processing speed of work queue.

Linked PRs
@jackcvr jackcvr added the type-feature A feature request or enhancement label Feb 18, 2023
@bedevere-bot bedevere-bot mentioned this issue Feb 18, 2023
Merged
@arhadthedev arhadthedev added the stdlib Python modules in the Lib dir label Feb 18, 2023
@jackcvr jackcvr changed the title Reduce IDLE semaphore calls Reduce IDLE semaphore calls in ThreadPoolExecutor Feb 18, 2023
@terryjreedy terryjreedy changed the title Reduce IDLE semaphore calls in ThreadPoolExecutor Reduce _idle_semaphore calls in ThreadPoolExecutor Feb 18, 2023
@terryjreedy
Copy link
Member

'IDLE' (capitalized) is the IDE that comes with CPython; it has nothing to do with inactivity ;-).

gpshead pushed a commit that referenced this issue May 26, 2023
@jackcvr @blurb-it @hauntsaninja
gh-102024: Reduced _idle_semaphore.release calls (#102025)
0242e9a 
Reduced _idle_semaphore.release calls in concurrent.futures.thread._worker
_idle_semaphore.release() is now only called if only work_queue is empty.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 26, 2023
@jackcvr @blurb-it
@hauntsaninja @miss-islington
pythongh-102024: Reduced _idle_semaphore.release calls (pythonGH-102025)
48d7818 
Reduced _idle_semaphore.release calls in concurrent.futures.thread._worker
_idle_semaphore.release() is now only called if only work_queue is empty.

---------

(cherry picked from commit 0242e9a)

Co-authored-by: Andrii Kuzmin <jack.cvr@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
@bedevere-bot bedevere-bot mentioned this issue May 26, 2023
Merged
@gpshead gpshead added the performance Performance or resource usage label May 26, 2023
@gpshead
Copy link
Member

gpshead commented May 26, 2023

thanks!

@gpshead gpshead closed this as completed May 26, 2023
gpshead pushed a commit that referenced this issue May 26, 2023
@miss-islington @jackcvr
@blurb-it @hauntsaninja
[3.12] gh-102024: Reduced _idle_semaphore.release calls (GH-102025) (#…
8010cef 
…104959)

gh-102024: Reduced _idle_semaphore.release calls (GH-102025)

Reduced _idle_semaphore.release calls in concurrent.futures.thread._worker
_idle_semaphore.release() is now only called if only work_queue is empty.

---------

(cherry picked from commit 0242e9a)

Co-authored-by: Andrii Kuzmin <jack.cvr@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
@erlend-aasland erlend-aasland mentioned this issue Jun 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
Threading issues 🧵
Status: Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@gpshead @jackcvr @arhadthedev @terryjreedy