AbstractInterpreter: enable selective pure/concrete eval for external AbstractInterpreter with overlayed method table

[aviatesk]

Built on top of #44511 and #44561, and solves JuliaGPU/GPUCompiler.jl#309.
This commit allows external AbstractInterpreter to selectively use
pure/concrete evals even if it uses an overlayed method table.
More specifically, such AbstractInterpreter can use pure/concrete evals
as far as any callees used in a call in question doesn't come from the
overlayed method table:

@test Base.return_types((), MTOverlayInterp()) do
    isbitstype(Int) ? nothing : missing
end == Any[Nothing]
Base.@assume_effects :terminates_globally function issue41694(x)
    res = 1
    1 < x < 20 || throw("bad")
    while x > 1
        res *= x
        x -= 1
    end
    return res
end
@test Base.return_types((), MTOverlayInterp()) do
    issue41694(3) == 6 ? nothing : missing
end == Any[Nothing]

In order to check if a call is tainted by any overlayed call, our effect
system now additionally tracks overlayed::Bool property. This effect
property is required to prevent concrete-eval in the following kind of situation:

strangesin(x) = sin(x)
@overlay OverlayedMT strangesin(x::Float64) = iszero(x) ? nothing : cos(x)
Base.@assume_effects :total totalcall(f, args...) = f(args...)
@test Base.return_types(; interp=MTOverlayInterp()) do
    # we need to disable partial pure/concrete evaluation when tainted by any overlayed call
    if totalcall(strangesin, 1.0) == cos(1.0)
        return nothing
    else
        return missing
    end
end |> only === Nothing

[maleadt]

I'm not familiar enough with this code to review, but it does fix the GPUCompiler issue and makes CUDA.jl tests pass on 1.9.

[aviatesk]

Ok, the entire PRs (this one and #44561) ended up being bigger than I expected, but I think this is ready for another review.

@nanosoldier runbenchmarks("inference", vs=":master")

[aviatesk]

For correctness, we need to look at the effect of the pure function being analyzed here and assert that it doesn't involve any overlayed method call within it.
But I think a pure function isn't really supposed to call a generic function (which may be overlayed) anyway, and so here I'd like to avoid trying hard to exclude such rare and invalid usecases and adding implementation complexity

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

[aviatesk]

Rebased to take in #44561 . I'm planning to merge this once I confirm CI runs correctly as Kristoffer and I want those PR to be included in the next beta release.

[Keno]

This is bigger than a UInt8 now.

[Keno]

In #43899 I changed this to a UInt32, so you may want to pick up those pieces.

[aviatesk]

Since e.overlayed is a Boolean property, I think this can be encoded within UInt8?

julia> let maxbits = 0x02
           (maxbits << 1) |
           (maxbits << 3) |
           (maxbits << 5) |
           (maxbits << 7) |
           true
       end |> Int
85

[Keno]

No, 4*2+1 is 9 bits. You're cutting of the top bit (in particular the top bit of the two bit unit you're shifting by 7).

[aviatesk]

Au, you're right. Maybe we can pick UInt16 instead though?

[Keno]

I think UInt32 is fine. I have at least two more effects I think would be interesting, so with you're extra bit, and the one I already have in that PR, we'd be out again. It's not a huge amount of memory.