Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine: fixed_location task groups #3787

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

colinthomas-z80
Copy link
Contributor

Proposed changes

This is an effort to detect groups of sequential, temp-file dependent tasks, and send them in batches to a single worker. Hopefully avoiding the overhead and complexity of maintaining fixed_location policy.

Post-change actions

Put an 'x' in the boxes that describe post-change actions that you have done.
The more 'x' ticked, the faster your changes are accepted by maintainers.

  • make test Run local tests prior to pushing.
  • make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
  • make lint Run lint on source code prior to pushing.
  • Manual Update Did you update the manual to reflect your changes, if appropriate? This action should be done after your changes are approved but not merged.
  • Type Labels Select github labels for the type of this change: bug, enhancement, etc.
  • Product Labels Select github labels for the product affected: TaskVine, Makeflow, etc.
  • PR RTM Mark your PR as ready to merge.

Additional comments

This section is dedicated to changes that are ambitious or complex and require substantial discussions. Feel free to start the ball rolling.

@colinthomas-z80
Copy link
Contributor Author

This currently works (from some testing i have done). It does some pretty expensive searches and is a bit invasive to the code. I wanted to open it up at this stage in case anyone has thoughts on the idea and implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant