Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine: workers without peer transfers and temp files #3830

Open
btovar opened this issue May 9, 2024 · 3 comments
Open

vine: workers without peer transfers and temp files #3830

btovar opened this issue May 9, 2024 · 3 comments

Comments

@btovar
Copy link
Member

btovar commented May 9, 2024

If a worker does not have a transfer port, should it be able to execute tasks with temporary files? In general I'd say no, but maybe there are cases where a single large worker is executing the workflow?

One possible issue right now is that a temp file may be recreated at many workers because the worker can't fetch it from each other (if I understood the code correctly).

@btovar btovar added the TaskVine label May 9, 2024
@dthain
Copy link
Member

dthain commented May 9, 2024

Hmm, under what conditions would the worker not have a port at all? Are you observing cases where listening on the port fails, or is this a situation where the user deliberately disables the transfer port?

@btovar
Copy link
Member Author

btovar commented May 9, 2024

Once in a while condor+docker give me a worker whose reported port can't be accessed by other workers.

@btovar
Copy link
Member Author

btovar commented May 9, 2024

Once in a while meaning one or two in 1000 workers, but enough to halt a workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants