precompilation error when using module with PythonCall.jl in another module

[kylebeggs]

Affects: PythonCall

Describe the bug
I have a module Foo which imports a Python package and defines some functions using this Python package. All tests pass and it works great. I have another module Bar which uses Foo. During precompilation of Bar I get this error:

Failed to precompile Bar [03231224-7cb2-45ca-b412-3b82d488d8bd] to "/home/kylebeggs/.julia/compiled/v1.9/Bar/jl_8mIASd".
hwloc/linux: Ignoring PCI device with non-16bit domain.
Pass --enable-32bits-pci-domain to configure to support such devices
(warning: it would break the library ABI, don't enable unless really needed).
ERROR: LoadError: InitError: LoadError: Evaluation into the closed module `Foo` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `Foo` with `eval` during precompilation - don't do this.

In Foo, I load the Python package and some user defined Python functions as

const MP = PythonCall.pynew()

function __init__()
    PythonCall.pycopy!(MP, pyimport("some_package"))
    include(joinpath(@__DIR__, "python-funcs.jl"))
    return nothing
end

Your system
Please provide detailed information about your system:

• Linux - PopOS 22.04
• Julia 1.9.2
• PythonCall v0.9.13
• CondaPkg v0.2.18

Julia Version 1.9.2
Commit e4ee485e909 (2023-07-05 09:39 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 20 × 12th Gen Intel(R) Core(TM) i7-12700H
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, alderlake)
  Threads: 8 on 20 virtual cores
Environment:
  LD_PRELOAD = /usr/lib/x86_64-linux-gnu/libstdc++.so.6
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 8
  JULIA_IMAGE_THREADS = 1

[cjdoris]

I don't think this is a problem with PythonCall, the error message is pretty clear:

ERROR: LoadError: InitError: LoadError: Evaluation into the closed module Foo breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating Foo with eval during precompilation - don't do this.

I suspect it's because you are calling include in __init__.

[kylebeggs]

Ok, thank you very much for the response. I will look into using include in __init__.

[kylebeggs]

Using include in __init__ is indeed the issue. This confuses me as I thought include simply pastes the code into that spot. Why would manually copy/pasting the code in include into __init__ fix this issue?

[cjdoris]

include doesn't literally paste the code in (after all, include is an ordinary function, and a function cannot do such a thing, only a macro can). Roughly speaking, include(file) does eval(Meta.parse(read(file, String))). If you do this at the top level of a module, it has the same effect as pasting the code into the module (because eval evaluates the code in the top-level scope of the module), but if you do it in a function, then it has all the same caveats as calling @eval from within a function.

[kylebeggs]

Ahh, I see, thank you very much for the response/explanation.