Turn Method Overwritten Error into a PrecompileError -- turning off caching

[vchuravy]

Fixes #52213

Overwritting methods during cache creation is currently not something that the system
can support and can lead to surprising, counter-intuitive and fatal errors.

In 1.10 we turned it from a warning to a strong error, with this PR it remains
a strong error, but the precompilation system recognizes it and essentially sets __precompile__(false)
for this module and all modules that depend on it.

Before:

julia> using OverwriteMethodError
[ Info: Precompiling OverwriteMethodError [top-level]
WARNING: Method definition +(Bool, Bool) in module Base at bool.jl:166 overwritten in module OverwriteMethodError at /home/vchuravy/src/julia2/OverwriteMethodError.jl:2.
ERROR: LoadError: Method overwriting is not permitted during Module precompile.
Stacktrace:
 [1] top-level scope
   @ ~/src/julia2/OverwriteMethodError.jl:2
 [2] include
   @ Base ./Base.jl:489 [inlined]
 [3] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
   @ Base ./loading.jl:2216
 [4] top-level scope
   @ stdin:3
in expression starting at /home/vchuravy/src/julia2/OverwriteMethodError.jl:1
in expression starting at stdin:3
ERROR: Failed to precompile OverwriteMethodError [top-level] to "/home/vchuravy/.julia/compiled/v1.10/jl_guiuCQ".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2462
  [3] compilecache
    @ Base ./loading.jl:2334 [inlined]
  [4] (::Base.var"#968#969"{Base.PkgId})()
    @ Base ./loading.jl:1968
  [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.0-rc1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
  [6] #mkpidlock#6
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.0-rc1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
  [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.0-rc1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
  [8] #invokelatest#2
    @ Base ./essentials.jl:889 [inlined]
  [9] invokelatest
    @ Base ./essentials.jl:884 [inlined]
 [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2977
 [11] maybe_cachefile_lock
    @ Base ./loading.jl:2974 [inlined]
 [12] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1964
 [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1806
 [14] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [15] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1797
 [17] macro expansion
    @ Base ./loading.jl:1784 [inlined]
 [18] macro expansion
    @ Base ./lock.jl:267 [inlined]
 [19] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1747
 [20] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [21] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [22] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1740

After:

julia> using OverwriteMethodError
┌ Info: Precompiling OverwriteMethodError [top-level]
└ @ Base loading.jl:2486
WARNING: Method definition +(Bool, Bool) in module Base at bool.jl:166 overwritten in module OverwriteMethodError at /home/vchuravy/src/julia2/OverwriteMethodError.jl:2.
ERROR: Method overwriting is not permitted during Module precompile.
┌ Info: Skipping precompilation since __precompile__(false). Importing OverwriteMethodError [top-level].
└ @ Base loading.jl:2084

[aplavin]

May I suggest adding something like

state of your packages may depend on their loading order

to the warning to make it easier to understand?

[PallHaraldsson]

That was quick and it's certainly better to have something shorter to be readable. But I see "Info", "WARNING" and "ERROR" for the same cause...

I'm not even sure yet if this should only be a WARNING, or an ERROR actually... If I trust you then it needs not error, by turning precompilation off, but I still think the root cause is the redefinition. If I'm wrong, can it be changed to just a WARNING?

I'm sympathetic to people wanting stuff "working" that did seemed to work before. Should this say something like:

WARNING, precompilation is turned off for module X, and this is a potential order-defendant ERROR, because of type-piracy, and in future versions of Julia this will be an ERROR.

I see Kristoffer gave thumbs down, so maybe the change you made or anything (I suggested) isn't in the cards. Could it ERROR by default, the status quo for the latest Julia, but be overridable somehow, maybe just with an ENV var to make it inconvenient... and not promise a CLI option for the future. Would that make @aplavin and @KristofferC happy?

[aplavin]

I'd be glad with any user-based workaround/flag that doesn't require package code changes. This alone would help dealing with the breakage in 1.10.

[maleadt]

FWIW, there's currently 214 packages (6% of all current failures) that fail daily PkgEval because of this error, so it would help alleviate some of the 1.11-related breakage.

[BioTurboNick]

Automatically doing the thing that would be required by a package author to make it work is wonderful. I would much prefer it to never error in 1.x. A warning that clearly spells out the implications would be a large improvement, and might well incentivize more package authors to find alternatives. Especially if users also understand the problem, and log issues to get the warning to go away. But in the meantime, everything still works.

I would agree with PallHaraldsson that avoiding three log messages for the same event would be better.

[IanButterworth]

What about a flag/env var to turn it back into a warning, which is mentioned in the error and is enabled in PkgEval

[IanButterworth]

For this to not be highly confusing I think the message for PrecompilableError needs tweaking.

Skipping precompilation since __precompile__(false)

because there won't be any __precompile__(false) instances.

[IanButterworth]

That type of confusion does already exist, but I think it would be unwise to add more places that throw PrecompilableError without fixing the message.

[KristofferC]

#51367

[vtjnash]

this is noisy during CI:

precompile                                       (1) |        started at 2023-11-25T02:42:26.767
WARNING: Method definition +(Bool, Bool) in module Base at bool.jl:166 overwritten in module OverwriteMethodError at C:\Users\julia\AppData\Local\Temp\jl_shtGYs\OverwriteMethodError.jl:2.
ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.

(needs @test_warn)