BUG: if PYTHON_JULIAPKG_PROJECT is set to existing Project, the project get's deleted

[schlichtanders]

via PythonCall/JuliaCall I came to use JuliaPkg

unfortunately sofar JuliaPkg creates more problems than it solves. I am using PythonCall on mybinder, however the JuliaPkg creates its own empty Project, not using the binder project.

Even worse: If setting PYTHON_JULIAPKG_PROJECT to the directory where the binder Project.toml lies, it overwrites the existing Project.toml file, deleting the previous environment and leaving JuliaPkg empty environment.

[marius311]

Also ran into this, yea the behavior of overwriting is really surprising and undesired. If a PYTHON_JULIAPKG_PROJECT is specified, can it just assume necessary packages are there, and just allow errors if they're not?

[cjdoris]

This is not a bug, it's a documented feature.

If you want JuliaPkg to leave the project alone, you can set PYTHON_JULIAPKG_OFFLINE=yes.

However a missing feature I am aware of is a lack of a "backends" mechanism similar to CondaPkg which allows a finer control over what pre-installed things get used.

[marius311]

Maybe I'm missing something but its hard to imagine a use-case for removing all packages in an existing Project.toml. Perhaps better would just be an error message saying to set PYTHON_JULIAPKG_OFFLINE=yes if you want to use an existing project.

[cjdoris]

The current assumption in JuliaPkg is that it "owns" the project that it uses, and is therefore free to add and remove packages as it requires to satisfy any dependencies. Additionally keeping around pre-installed packages is more difficult to implement.

The env var is simply a mechanism to change where that project gets put, in case you want dependencies installed on a different filesystem for example.

As I said, a backend which lets you use a pre-installed project without modifying it could be added, but it's not done yet.

[sylvaticus]

This happens even if you first add Julia packages by seval julia code as jl.seval("Pkg.add("Foo")"). This creates an entry for Foo on the Project.toml in the current environment.
If however then you use pyjuliapkg.add("Goo", "goo_uuid") and pyjuliapkg.resolve(), this will flush the environment information from juliapkg.json to Project.toml overriting what's there, and without warning.
Perhaps attempt a merging or an option (default off) to force the flush in case of divergence ?

Also one question: which are the advantages of adding a Julia package using pyjuliapkg instead of evaluating the Julia Pkg.add() code ?