-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem with starting multiple Julia process on a cluster at the same time #31953
Comments
I added 0.1 sleep time between starting each process. This seems to have avoided the problem. |
related issue #30174 |
Hi, @newptcai , I have similar issue when I deployed julia in HPC for running jobs. Sometimes the package ``KernelDensity'' can be loaded in a worker correctly but sometimes failed. How did you add sleep time between starting each process? Thanks in advance. |
This is still an issue on 1.7.2: My guess as to what is happening is that one process is attempting to load a precompile package, while another deletes it from the cache. In particular, it gets deleted between this line There are a couple of options as far as I can see:
|
master now has pidlocking around the precompilation process |
I am trying to start multiple Julia processes on a cluster at the same time using a Python script (parallel-ssh). I noticed that, a few of these processes will fail to start, with the following errors.
The cluster has a shared network file system. This could be the source of the issue. But this has not been any problem when previously everything is done in Python.
The text was updated successfully, but these errors were encountered: