Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using PythonCall with Distributed #351

Closed
martinmestre opened this issue Aug 24, 2023 · 2 comments
Closed

Using PythonCall with Distributed #351

martinmestre opened this issue Aug 24, 2023 · 2 comments
Labels
question Further information is requested

Comments

@martinmestre
Copy link

Hello,
I would like to know which modification should I do to the following piece of code in order to make PythonCall compatible with Distributed.jl. The problem arises when I have more than one process.

 @everywhere begin
    pyimport("sys")."path".append("")
    # pushfirst!(pyconvert(Vector{String}, pyimport("sys")."path"), "")
    importLib = pyimport("importlib")
    stream = pyimport("stream")
    potentials = pyimport("potential_classes")
    u = pyimport("astropy.units")
    # importLib.reload(stream)
    # importLib.reload(potentials)
end  

The output is:
ERROR: Python: Julia: an error occurred while setting an error
I guess I am making something wrong with the first line in the begin block. The second (commmented) line works with PyCall. If I avoid both lines, then the program doen't recognize the Python packages "stream".
Thank you very much.
Best

@martinmestre martinmestre added the question Further information is requested label Aug 24, 2023
@cjdoris
Copy link
Collaborator

cjdoris commented Aug 31, 2023

Can you post a full MWE please? i.e. a complete self contained piece of code with all imports etc.

Does the error occur on that first line? i.e. does it still error if you comment out the rest?

How are you launching Julia?

@martinmestre
Copy link
Author

Hello, thank for your answer. I found out that there was an error in the astropy version I was using in the cluster. That is why I thought that it was a problem with distributed or slurm.
The line pyimport("sys")."path".append("") works well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants