Strange "missing from cache"/"does not support precompilation"/... errors #52132

aplavin · 2023-11-12T13:57:44Z

Reproducing in a clean env & depot, latest Julia (1.9.3; upd same on 1.9.4 and 1.10):

pkg> add AccessorsExtra
julia> using AccessorsExtra

┌ LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5]
│  ┌ Warning: Module LinearAlgebraExt with build ID ffffffff-ffff-ffff-0001-6be4cd7b7f3f is missing from the cache.
│  │ This may mean LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension LinearAlgebraExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace: <...>
└  
┌ TestExt [69cf5fdc-2bf6-5cf1-837e-a22f5027a90f]
│  ┌ Warning: Module TestExt with build ID ffffffff-ffff-ffff-0001-6be4cd7ce9e0 is missing from the cache.
│  │ This may mean TestExt [69cf5fdc-2bf6-5cf1-837e-a22f5027a90f] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension TestExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace: <...>
└  
[ Info: Precompiling LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5]
┌ Warning: Module LinearAlgebraExt with build ID ffffffff-ffff-ffff-0001-6be52894ee31 is missing from the cache.
│ This may mean LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1793
┌ Error: Error during loading of extension LinearAlgebraExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace: <...>

Still, it seems like code from these two extensions is actually loaded and works.
I don't have __precompile__(false) anywhere in that package, all modules should support precompilation. What these errors and warnings actually mean, and are they bugs in Julia or in package code?

The text was updated successfully, but these errors were encountered:

IanButterworth · 2023-11-13T01:00:57Z

Does this reproduce on master?

aplavin · 2023-11-16T18:04:54Z

Not sure about master, but I see the same on 1.10rc.

IanButterworth · 2023-11-16T18:11:59Z

1.9.4 or master would be helpful

aplavin · 2023-11-16T18:23:35Z

1.9.4 same

aplavin · 2023-11-29T02:41:25Z

Any ideas why these messages would appear?..
And what are they – infos, warnings, or errors? :) All three labels are printed in sequence.

aplavin · 2023-12-08T18:56:36Z

From a slack discussion (@lassepe), these warnings/errors seem to occur when Foo is a weakdep of MyPkg, but is a strong dep of another dependency of MyPkg. If that's the case, this does seem a bug in Julia not in packages.

aplavin · 2024-01-09T13:44:15Z

Any updates on this?.. Everything works, but big "errors"/warnings look scary to users (:

IanButterworth · 2024-01-09T13:58:51Z

Do you see this on 1.10.0 & master? If so please share the output on those

KristofferC · 2024-01-09T14:04:00Z

(I can repro this on 1.10)

I wonder if this could be related to all the method overwrites in AccessorsExtra:

┌ AccessorsExtra → LinearAlgebraExt
│  WARNING: Method definition (::Type{Accessors.IndexLens{I} where I<:Tuple})(Tuple{Accessors.Elements}) in module Accessors at C:\Users\Kristoffer\.julia\packages\Accessors\ss6Uj\src\sugar.jl:463 overwritten in module AccessorsExtra at C:\Users\Kristoffer\.julia\packages\AccessorsExtra\h0y2R\src\AccessorsExtra.jl:87.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└

which causes loading the AccessorsExtra package to fail at precompilation and as a fallback it loads without a precomple cache (see #52214) which then reloads the extensions etc.

AcessorsExtra has stuff like:

# some piracy:
Accessors.IndexLens(::Tuple{Elements}) = Elements()
Accessors.IndexLens(::Tuple{Properties}) = Properties()
Accessors._shortstring(prev, o::Elements) = "$prev[∗]"

...

# some piracy - should upstream:
getall(obj::AbstractDict, ::Elements) = collect(obj)
getall(obj::AbstractSet, ::Elements) = collect(obj)

Edit: I removed what I think are all method overwrites and it does not seem to fix it what I can see.

KristofferC · 2024-01-09T14:16:22Z

This repros with only Accessors for me (also, does not require Pkg.precompile to repro):

julia> ENV["JULIA_PKG_PRECOMPILE_AUTO"] = 0
0

julia> Base.PKG_PRECOMPILE_HOOK[] = Returns(nothing)
Returns{Nothing}(nothing)

(jl_4ZbHIu) pkg> add Accessors
...

julia> using Accessors
[ Info: Precompiling Accessors [7d9f7c33-5ae7-4f3b-8dc6-eff91059b697]
[ Info: Precompiling AccessorsDatesExt [308068fd-b978-57fd-b78f-2a52fb63dba4]
┌ Warning: Module AccessorsDatesExt with build ID ffffffff-ffff-ffff-0001-6544d38c5b81 is missing from the cache.
│ This may mean AccessorsDatesExt [308068fd-b978-57fd-b78f-2a52fb63dba4] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1942
┌ Error: Error during loading of extension AccessorsDatesExt of Accessors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace:
│     ...
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295
┌ Warning: Module AccessorsDatesExt with build ID ffffffff-ffff-ffff-0001-6544d38c5b81 is missing from the cache.
│ This may mean AccessorsDatesExt [308068fd-b978-57fd-b78f-2a52fb63dba4] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1942
┌ Error: Error during loading of extension AccessorsDatesExt of Accessors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace:
│     ...
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295
┌ Warning: Module AccessorsLinearAlgebraExt with build ID ffffffff-ffff-ffff-0001-6544f41e6c6d is missing from the cache.
│ This may mean AccessorsLinearAlgebraExt [6d540e18-2ad7-59a5-acf8-1761f4011e1b] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1942
┌ Error: Error during loading of extension AccessorsLinearAlgebraExt of Accessors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace:
│     ...
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295

IanButterworth · 2024-01-09T14:18:05Z

I wonder if making multiple extensions that each depend on a different packages in the sysimage is racy

dalum · 2024-01-09T15:02:58Z

I just updated to 1.10.0 (from 1.8.5) and have started seeing this, and it seems to be causing sysimage builds with PackageCompiler to fail in our pipeline with the error:

ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
ERROR: LoadError: Declaring __precompile__(false) is not allowed in files that are being precompiled.

Trying to see if I can make an MWE.

aplavin · 2024-01-09T15:30:50Z

Do you see this on 1.10.0 & master? If so please share the output on those

Yes, same output on 1.10.

I wonder if this could be related to all the method overwrites in AccessorsExtra:

Nope, not related.
Btw method overrides there are a historical remnant, these methods were just upstreamed to Accessors itself. They weren't method overrides when I opened this issue.

This repros with only Accessors for me

Recently we moved to "make everything a weakdep when possible" in Accessors, making this issue manifest for that package alone as well.

jw3126 · 2024-01-09T17:29:54Z

This repros with only Accessors for me (also, does not require Pkg.precompile to repro):

FYI we made things stong deps again in Accessors, to circumvent this issue. People need Accessors.jl v0.1.34 to reproduce.

KristofferC · 2024-01-09T19:23:41Z

ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.

I don't think this is related to the issue here.

KristofferC · 2024-01-09T21:31:36Z

Just a bit of an update here. We (me and @IanButterworth) think this issue has to do with adding extensions to packages that are in the sysimage (most stdlibs on 1.10 and e.g. LinearAlgebra on 1.11).

So removing extensions for the stdlibs might enough (and worth trying). cc @jw3126

KristofferC · 2024-01-09T22:06:04Z

Here is a somewhat untested patch that tries to avoid the Base.loaded_modules for checking if a dependency has been loaded and instead rely on require calls (thereby trying to avoid the issue of what is and isn't in the sysimage):

diff --git a/base/Base.jl b/base/Base.jl
index 0ca13265ad..98803cf29f 100644
--- a/base/Base.jl
+++ b/base/Base.jl
@@ -627,6 +627,7 @@ function __init__()
     init_load_path()
     init_active_project()
     append!(empty!(_sysimage_modules), keys(loaded_modules))
+    empty!(required_modules)
     if haskey(ENV, "JULIA_MAX_NUM_PRECOMPILE_FILES")
         MAX_NUM_PRECOMPILE_FILES[] = parse(Int, ENV["JULIA_MAX_NUM_PRECOMPILE_FILES"])
     end
diff --git a/base/loading.jl b/base/loading.jl
index 97aa9977a7..a87df18902 100644
--- a/base/loading.jl
+++ b/base/loading.jl
@@ -1343,7 +1343,7 @@ function _insert_extension_triggers(parent::PkgId, extensions::Dict{String, Any}
             # TODO: Better error message if this lookup fails?
             uuid_trigger = UUID(weakdeps[trigger]::String)
             trigger_id = PkgId(uuid_trigger, trigger)
-            if !haskey(Base.loaded_modules, trigger_id) || haskey(package_locks, trigger_id)
+            if !(trigger_id in required_modules) || haskey(package_locks, trigger_id)
                 trigger1 = get!(Vector{ExtensionId}, EXT_DORMITORY, trigger_id)
                 push!(trigger1, gid)
             else
@@ -1965,8 +1965,11 @@ function _require_prelocked(uuidkey::PkgId, env=nothing)
     end
 end
 
+const required_modules = Set{PkgId}()
 function __require_prelocked(uuidkey::PkgId, env=nothing)
     assert_havelock(require_lock)
+    first_require = !(uuidkey in required_modules)
+    push!(required_modules, uuidkey)
     if !root_module_exists(uuidkey)
         newm = _require(uuidkey, env)
         if newm === nothing
@@ -1977,6 +1980,7 @@ function __require_prelocked(uuidkey::PkgId, env=nothing)
         # After successfully loading, notify downstream consumers
         run_package_callbacks(uuidkey)
     else
+        first_require && run_package_callbacks(uuidkey)
         warn_if_already_loaded_different(uuidkey)
         newm = root_module(uuidkey)
     end

With this I can precompile Accessors.jl with no problems, but I literally haven't tried anything else than that.

Edit: I don't think the above is completely correct.

aplavin · 2024-01-09T22:06:40Z

So removing extensions for the stdlibs might enough (and worth trying)

That's what we did in Accessors for now.
But note that packages do make extensions for stdlibs when it makes sense, eg https://github.com/JuliaLang/Compat.jl/blob/master/Project.toml. I guess it works fine for them?

KristofferC · 2024-01-09T22:09:29Z

They only have one extension for an stdlib. If they added another one, they might encounter the same problem.

The issue AFAIU is that:

Accessors is loaded
LinearAlgebraExt starts to precompile (because LinearAlgebra is in the sysimage)
This loads Accessors, which loads Test (because of Accessors → InverseFunctions → Test)
TestExt starts to precompile
This loads Accessors which loads LinearAlgebraExt(!!!) again because LinearAlgebra is in the sysimage.
This is a cycle.

If you only have one extension there is no way to "loop back" to the first extension as in the example above.

atbug · 2024-01-19T21:37:07Z

this issue has to do with adding extensions to packages that are in the sysimage

Maybe not? I see such errors with this package:

[deps]
ComputedFieldTypes = "459fdd68-db75-56b8-8c15-d717a790f88e"
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
UniformGrids = "3d03f9f6-d0fb-45b8-9ee8-5e9698a0a906"

[weakdeps]
Cells = "c368a612-970d-47d0-a74e-c409b3f7b97e"
SiestaInterface = "820187e7-b133-44e4-ad7a-78036505f177"

[extensions]
OrbitalLatticesSiestaExt = ["Cells", "SiestaInterface"]
OrbitalsSiestaExt = ["SiestaInterface"]

[extras]
Cells = "c368a612-970d-47d0-a74e-c409b3f7b97e"
SiestaInterface = "820187e7-b133-44e4-ad7a-78036505f177"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test", "Cells", "SiestaInterface"]

Many of the packages above are currently private, unfortunately. Let me know if more information is needed.

Edit: I am not sure how to reproduce the error. Will edit next time I see the error.

Edit: Stack trace:

Precompiling project...
  2 dependencies successfully precompiled in 10 seconds. 90 already precompiled.
  1 dependency had output during precompilation:
┌ TightBinding → OrbitalsSiestaExt
│  ┌ Warning: Module OrbitalsSiestaExt with build ID ffffffff-ffff-ffff-0000-3f0623d76494 is missing from the cache.
│  │ This may mean OrbitalsSiestaExt [6d573fe1-6245-5e66-b3c2-3070708ad8ae] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1942
│  ┌ Error: Error during loading of extension OrbitalsSiestaExt of TightBinding, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1946
│  │      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1806
│  │      [3] #invoke_in_world#3
│  │        @ Base ./essentials.jl:921 [inlined]
│  │      [4] invoke_in_world
│  │        @ Base ./essentials.jl:918 [inlined]
│  │      [5] _require_prelocked
│  │        @ Base ./loading.jl:1797 [inlined]
│  │      [6] _require_prelocked
│  │        @ Base ./loading.jl:1796 [inlined]
│  │      [7] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1289
│  │      [8] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1324
│  │      [9] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1158
│  │     [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1813
│  │     [11] #invoke_in_world#3
│  │        @ Base ./essentials.jl:921 [inlined]
│  │     [12] invoke_in_world
│  │        @ Base ./essentials.jl:918 [inlined]
│  │     [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1797
│  │     [14] macro expansion
│  │        @ Base ./loading.jl:1784 [inlined]
│  │     [15] macro expansion
│  │        @ Base ./lock.jl:267 [inlined]
│  │     [16] __require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1747
│  │     [17] #invoke_in_world#3
│  │        @ Base ./essentials.jl:921 [inlined]
│  │     [18] invoke_in_world
│  │        @ Base ./essentials.jl:918 [inlined]
│  │     [19] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1740
│  │     [20] include
│  │        @ Base ./Base.jl:495 [inlined]
│  │     [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2216
│  │     [22] top-level scope
│  │        @ stdin:3
│  │     [23] eval
│  │        @ Core ./boot.jl:385 [inlined]
│  │     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:2070
│  │     [25] include_string
│  │        @ Base ./loading.jl:2080 [inlined]
│  │     [26] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:316
│  │     [27] _start()
│  │        @ Base ./client.jl:552
│  └ @ Base loading.jl:1295
└

…re in the sysimage (#52841) When triggers of extension are in the sysimage it is easy to end up with cycles in package loading. Say we have a package A with exts BExt and CExt and say that B and C is in the sysimage. - Upon loading A, we will immidiately start to precompile BExt (because the trigger B is "loaded" by virtue of being in the sysimage). - BExt will load A which will cause CExt to start precompiling (again because C is in the sysimage). - CExt will load A which will now cause BExt to start loading and we get a cycle. This is fixed in this PR by instead of looking at what modules are loaded, we look at what modules are actually `require`d and only use that to drive the loading of extensions. Fixes #52132.

…re in the sysimage (#52841) When triggers of extension are in the sysimage it is easy to end up with cycles in package loading. Say we have a package A with exts BExt and CExt and say that B and C is in the sysimage. - Upon loading A, we will immidiately start to precompile BExt (because the trigger B is "loaded" by virtue of being in the sysimage). - BExt will load A which will cause CExt to start precompiling (again because C is in the sysimage). - CExt will load A which will now cause BExt to start loading and we get a cycle. This is fixed in this PR by instead of looking at what modules are loaded, we look at what modules are actually `require`d and only use that to drive the loading of extensions. Fixes #52132. (cherry picked from commit 08d229f)

…re in the sysimage (JuliaLang#52841) When triggers of extension are in the sysimage it is easy to end up with cycles in package loading. Say we have a package A with exts BExt and CExt and say that B and C is in the sysimage. - Upon loading A, we will immidiately start to precompile BExt (because the trigger B is "loaded" by virtue of being in the sysimage). - BExt will load A which will cause CExt to start precompiling (again because C is in the sysimage). - CExt will load A which will now cause BExt to start loading and we get a cycle. This is fixed in this PR by instead of looking at what modules are loaded, we look at what modules are actually `require`d and only use that to drive the loading of extensions. Fixes JuliaLang#52132. (cherry picked from commit 08d229f)

aplavin · 2024-02-17T14:18:37Z

This issue is marked as closed, but still reproduces on 1.10.1, at least partially. @KristofferC

Output

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.1 (2024-02-13)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.10) pkg> activate --temp
  Activating new project at `/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_Jy6kID`

(jl_Jy6kID) pkg> add AccessorsExtra
   Resolving package versions...
    Updating `/private/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_Jy6kID/Project.toml`
  [33016aad] + AccessorsExtra v0.1.68
    Updating `/private/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_Jy6kID/Manifest.toml`
  [7d9f7c33] + Accessors v0.1.35
  [33016aad] + AccessorsExtra v0.1.68
  [a33af91c] + CompositionsBase v0.1.2
  [187b0558] + ConstructionBase v1.5.4
  [02685ad9] + DataPipes v0.3.14
  [3587e190] + InverseFunctions v0.1.12
  [1914dd2f] + MacroTools v0.5.13
  [189a3867] + Reexport v1.2.2
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [b77e0a4c] + InteractiveUtils
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [de0858da] + Printf
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [8dfed614] + Test
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.1.0+0
  [4536629a] + OpenBLAS_jll v0.3.23+4
  [8e850b90] + libblastrampoline_jll v5.8.0+1
Precompiling project...
  3 dependencies successfully precompiled in 4 seconds. 9 already precompiled.
  1 dependency had output during precompilation:
┌ AccessorsExtra → LinearAlgebraExt
│  ┌ Warning: Module LinearAlgebraExt with build ID ffffffff-ffff-ffff-0003-d7c6509dfd87 is missing from the cache.
│  │ This may mean LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1948
│  ┌ Error: Error during loading of extension LinearAlgebraExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1952
│  │      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1812
│  │      [3] #invoke_in_world#3
│  │        @ ./essentials.jl:926 [inlined]
│  │      [4] invoke_in_world
│  │        @ ./essentials.jl:923 [inlined]
│  │      [5] _require_prelocked
│  │        @ ./loading.jl:1803 [inlined]
│  │      [6] _require_prelocked
│  │        @ ./loading.jl:1802 [inlined]
│  │      [7] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1295
│  │      [8] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1330
│  │      [9] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1164
│  │     [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1819
│  │     [11] #invoke_in_world#3
│  │        @ ./essentials.jl:926 [inlined]
│  │     [12] invoke_in_world
│  │        @ ./essentials.jl:923 [inlined]
│  │     [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1803
│  │     [14] macro expansion
│  │        @ ./loading.jl:1790 [inlined]
│  │     [15] macro expansion
│  │        @ ./lock.jl:267 [inlined]
│  │     [16] __require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1753
│  │     [17] #invoke_in_world#3
│  │        @ ./essentials.jl:926 [inlined]
│  │     [18] invoke_in_world
│  │        @ ./essentials.jl:923 [inlined]
│  │     [19] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1746
│  │     [20] include
│  │        @ ./Base.jl:495 [inlined]
│  │     [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2222
│  │     [22] top-level scope
│  │        @ stdin:3
│  │     [23] eval
│  │        @ ./boot.jl:385 [inlined]
│  │     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:2076
│  │     [25] include_string
│  │        @ ./loading.jl:2086 [inlined]
│  │     [26] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:316
│  │     [27] _start()
│  │        @ Base ./client.jl:552
│  └ @ Base loading.jl:1301

vtjnash · 2024-02-17T14:53:25Z

AFAIK, #52841 was only intended to fix some of this issue / mitigate this bug in some cases, but not actually to close it?

vtjnash · 2024-02-17T14:54:14Z

The relevant issue for other examples similar to this is #52511

…re in the sysimage (JuliaLang#52841) When triggers of extension are in the sysimage it is easy to end up with cycles in package loading. Say we have a package A with exts BExt and CExt and say that B and C is in the sysimage. - Upon loading A, we will immidiately start to precompile BExt (because the trigger B is "loaded" by virtue of being in the sysimage). - BExt will load A which will cause CExt to start precompiling (again because C is in the sysimage). - CExt will load A which will now cause BExt to start loading and we get a cycle. This is fixed in this PR by instead of looking at what modules are loaded, we look at what modules are actually `require`d and only use that to drive the loading of extensions. Fixes JuliaLang#52132. (cherry picked from commit 08d229f)

aplavin mentioned this issue Jan 9, 2024

Errors in extensions during precompilation JuliaObjects/Accessors.jl#127

Closed

KristofferC added bug Indicates an unexpected problem or unintended behavior compiler:precompilation Precompilation of modules labels Jan 9, 2024

KristofferC mentioned this issue Jan 10, 2024

Extensions: make loading of extensions independent of what packages are in the sysimage #52841

Merged

lassepe mentioned this issue Jan 17, 2024

Extensions: pre-compilation fails if more than one extension #52511

Open

kpobrien mentioned this issue Jan 21, 2024

precompilation errors with SymbolicsForwardDiffExt and SymbolicsPreallocationToolsExt JuliaSymbolics/Symbolics.jl#1042

Closed

IanButterworth added the package extensions label Jan 24, 2024

KristofferC closed this as completed in #52841 Jan 29, 2024

aplavin mentioned this issue May 28, 2024

Naming a package extension "REPLExt" breaks loading JuliaLang/Pkg.jl#3906

Closed

devmotion mentioned this issue Aug 13, 2024

Make Test a weak dependency JuliaMath/InverseFunctions.jl#52

Merged

jw3126 mentioned this issue Aug 18, 2024

Don't make Test a dependency JuliaObjects/Accessors.jl#164

Closed

KristofferC mentioned this issue Oct 17, 2024

1.11.1: breaking change in extension loading #56204

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Strange "missing from cache"/"does not support precompilation"/... errors #52132

Strange "missing from cache"/"does not support precompilation"/... errors #52132

aplavin commented Nov 12, 2023 •

edited

Loading

IanButterworth commented Nov 13, 2023

aplavin commented Nov 16, 2023

IanButterworth commented Nov 16, 2023

aplavin commented Nov 16, 2023

aplavin commented Nov 29, 2023

aplavin commented Dec 8, 2023

aplavin commented Jan 9, 2024

IanButterworth commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

IanButterworth commented Jan 9, 2024

dalum commented Jan 9, 2024 •

edited

Loading

aplavin commented Jan 9, 2024 •

edited

Loading

jw3126 commented Jan 9, 2024

KristofferC commented Jan 9, 2024

KristofferC commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

aplavin commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

atbug commented Jan 19, 2024 •

edited

Loading

aplavin commented Feb 17, 2024

vtjnash commented Feb 17, 2024

vtjnash commented Feb 17, 2024

Strange "missing from cache"/"does not support precompilation"/... errors #52132

Strange "missing from cache"/"does not support precompilation"/... errors #52132

Comments

aplavin commented Nov 12, 2023 • edited Loading

IanButterworth commented Nov 13, 2023

aplavin commented Nov 16, 2023

IanButterworth commented Nov 16, 2023

aplavin commented Nov 16, 2023

aplavin commented Nov 29, 2023

aplavin commented Dec 8, 2023

aplavin commented Jan 9, 2024

IanButterworth commented Jan 9, 2024 • edited Loading

KristofferC commented Jan 9, 2024 • edited Loading

KristofferC commented Jan 9, 2024 • edited Loading

IanButterworth commented Jan 9, 2024

dalum commented Jan 9, 2024 • edited Loading

aplavin commented Jan 9, 2024 • edited Loading

jw3126 commented Jan 9, 2024

KristofferC commented Jan 9, 2024

KristofferC commented Jan 9, 2024 • edited Loading

KristofferC commented Jan 9, 2024 • edited Loading

aplavin commented Jan 9, 2024 • edited Loading

KristofferC commented Jan 9, 2024 • edited Loading

atbug commented Jan 19, 2024 • edited Loading

aplavin commented Feb 17, 2024

vtjnash commented Feb 17, 2024

vtjnash commented Feb 17, 2024

aplavin commented Nov 12, 2023 •

edited

Loading

IanButterworth commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

dalum commented Jan 9, 2024 •

edited

Loading

aplavin commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

aplavin commented Jan 9, 2024 •

edited

Loading

KristofferC commented Jan 9, 2024 •

edited

Loading

atbug commented Jan 19, 2024 •

edited

Loading