Stuck compiling function

[blegat]

The Julia compiler seems to get stuck when compiling the MathOptInterface add_constraint.
I have added a print inside the function and it is never called which suggests that it's the compiler itself that gets in an infinite loop.
The bug occurs in Julia v1.0, Julia v1.1 and Julia master

Julia Version 1.3.0-DEV.293
Commit 24497e8f53* (2019-05-28 13:40 UTC)

To reproduce it, use https://github.com/blegat/MOIStuck

[mlubin]

Bump @vtjnash @Keno. This compiler hang is blocking a new feature from JuMP. Are you able to take a look?

[JeffBezanson]

Thanks for the nice reproducer repo. When I run it on master it runs for 36 seconds, ending with this output:

Calling
Called
ERROR: LoadError: UndefVarError: bridge_type not defined
Stacktrace:
 [1] concrete_bridge_type(::Main.Bridges.BridgeOptimizer{Model{Float64}}, ::Type{Main.Bridges.MathOptInterface.VectorOfVariables}, ::Type{Main.Bridges.MathOptInterface.RootDetConeTriangle}) at /home/jeff/src/MOIStuck/src/Bridges/Bridges.jl:32
 [2] add_constraint(::Main.Bridges.BridgeOptimizer{Model{Float64}}, ::Main.Bridges.MathOptInterface.VectorOfVariables, ::Main.Bridges.MathOptInterface.RootDetConeTriangle) at /home/jeff/src/MOIStuck/src/Bridges/Bridges.jl:12
 [3] top-level scope at /home/jeff/src/MOIStuck/bug.jl:9

[blegat]

It seems at some point my timeout was too short and I simplified too much. I have bisected when I started to simplify too much and rolled back the master branch to a point where it is still failing. With MOIStuck master you should now have:

$ bash bug.sh 
Calling
bug.sh: line 1:  7475 Killed                  timeout -s 9 200 /home/blegat/git/julia-master/julia --color=yes bug.jl

real	3m20.001s
user	0m0.001s
sys	0m0.000s

[blegat]

The bug seems to affect MathOptInterface v0.9 (no idea whether earlier versions of MOI are also affected) with Julia v1.2.0-rc3.
The tests of the Bridges submodule never terminate on travis for 1.2 for MOI master (which is the same as MOI v0.9 as it has just been released):
https://travis-ci.org/JuliaOpt/MathOptInterface.jl/jobs/571538678#L231
but it runs fine with Julia v1.0 and v1.1.
I would bet that the bug is also present in v1.0 and v1.1 though as noticed in the first post of this issue.
By the way, rebasing the PR jump-dev/MathOptInterface.jl#751 where this bug was first encountered, makes the bug disappear because the @testset causing the bug was removed (for unrelated reason) but it is now also causing AppVeyor to fail for 32 bits on Julia v1.0:
https://ci.appveyor.com/project/JuliaOpt/mathoptinterface-jl/builds/26664620/job/p8tfm7qi7nu3vkvn
I have really no idea what in MOI triggers this bug and playing with MOIStuck didn't help me figure out a workaround so it would save us if it could be fixed before the release of Julia v1.2.

[mlubin]

Bump. This issue is now blocking the release of JuMP 0.20 because we're experiencing the hanging pretty consistently on Julia 1.2 (e.g., https://travis-ci.org/JuliaOpt/JuMP.jl/jobs/574225647).

[blegat]

I reduced the hanging tests of JuMP and MOI into the following:

using MathOptInterface
const MOI = MathOptInterface

MOI.Utilities.@model(Model, (), (), (), (), (), (), (), ())
MOI.supports_constraint(::Model{T}, ::Type{MOI.SingleVariable}, ::Type{MOI.GreaterThan{T}}) where {T} = false
MOI.supports_constraint(::Model{T}, ::Type{MOI.SingleVariable}, ::Type{MOI.LessThan{T}}) where {T} = false

model = MOI.Bridges.LazyBridgeOptimizer(Model{Float64}())
set = MOI.LessThan(1.0)
println("Calling")
MOI.add_constrained_variable(model, set)
println("Done")

With Julia master and MOI v0.9, the compiler hangs while compiling MOI.add_constrained_variable.

[odow]

I believe we have a temporary fix for this: jump-dev/MathOptInterface.jl#834

We needed to define two fallback methods. The weird thing is, these methods will never be called. (The errors are never triggered, if you add a print, it's never printed, and codecov is 0%.)

A common pattern in the code is

struct X end
foo_supported(::X) = true
foo(::X) = 1

struct Y end
foo_supported(::Y) = false
bar(::Any) = 2

function main(x)
    if foo_supported(x)
        return foo(x)
    else
        return bar(x)
    end
end

The fix is equivalent to defining

foo(::Y) = error("foo not defined for Y")

But it seems that this is just a MethodError...

I haven't managed to find a simple reproducible example that doesn't require MathOptInterface.jl#master.

Weirder still, if you @generate these methods, whether they get generated appears to depend on whether you add Core.println("Foo: ", x) or just Core.println("Foo: "):
jump-dev/MathOptInterface.jl#834 (comment)

I'm a little lost with what is going on, so the best argument I can come up with is that the compiler "speculatively compiles" the branch that we know we will not take at run-time, and that for some reason it doesn't trigger a MethodError but goes down some rabbit hole.

I'd appreciate any compiler team insight.

Also: Julia 1.2 is significantly slower than 1.1: https://travis-ci.org/JuliaOpt/MathOptInterface.jl/builds/575028355?utm_source=github_status&utm_medium=notification

[vtjnash]

Seems like an old issue. Open a new one if you come across problems again.