Add job group execution limit option #4457
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed change
Replace
process_lock
decorator job decorator by adding a newJOB_GROUP
execution limitation option. This option allows a thread to grab the lock for the group and then freely make calls to methods which share this common lock while still locking other callers out. Instead of having public methods using a lock and private methods that route around the lock to allow one method to call another after obtaining the singular lock on the class.Lock ownership is tracked via a contextvar which is cloned per asyncio task. This way we can easily determine what task has grabbed hold of a job group lock and which haven't.
In terms of one job calling another, each job defines what it has access to irrespective of its parent. So if a job in a job group calls another job in that job group, that works and the chain can continue. If a job calls another job in a different job group or not in a job group at all then the inner job cannot make more calls in the original job group until control returns to the parent.
This is the start of #1946 . It implements job groups and leverages them. Progress/stage options exist but are not currently used and there is no way to list all running jobs with the API. Those things can come in a subsequent PR.
Type of change
Additional information
Checklist
black --fast supervisor tests
)If API endpoints of add-on configuration are added/changed: