Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

effects: allow override of :nonoverlayed effect bit (and rename it to :native_executable) #51080

Closed
wants to merge 2 commits into from

Conversation

aviatesk
Copy link
Member

@aviatesk aviatesk commented Aug 28, 2023

Certain external AbstractInterpreters, such as GPUCompiler.jl, have
long sought the ability to allow concrete evaluation for specific
overlay-ed methods to achieve optimal inference accuracy. This is
currently not permitted, although it should be safe when an overlay-ed
method has the same semantics as the original method, and its result can
be safely replaced with the result of the original method. Refer to
JuliaGPU/GPUCompiler.jl#384 for more examples.

To address this issue, this commit introduces the capability to override
the :nonoverlayed effect bit using @assume_effects. With the
enhancements in PR #51078, this override behaves similarly to other
effect bits.

However, it now seems awkward to annotate a method with Base.@assume_effects :nonoverlayed
when it is actually marked with @overlay. Thus the second commit
replaces :nonoverlayed with (hopefully) more appropriate terminology,
:native_executable. This terminology is intended to indicate that the
method is concrete-evaluable. More specifically, it means that
(quoting from the new Base.@assume_effects docstring):

The :native_executable setting asserts that this method can be executed using Julia's
native compiler and runtime. Currently this particularly implies that any methods defined
in an [OverlayMethodTable](@ref Core.Compiler.OverlayMethodTable) (including this method in
question) are never be called during executing the method. However, it is worth noting that
it is safe to annotate [@overlay](@ref Base.Experimental.@overlay) method as
:native_executable when the overlay-ed method has the same semantics as the original
method and its result can safely be replaced with the result of the original method.

Consequently, external AbstractInterpreters can utilize
this feature to permit concrete evaluation for annotated overlay-ed
methods, e.g.

Base.@assume_effects :foldable function fact(x::Int)
    1 < x < 20 || error("x is too big")
    return factorial(x)
end

@overlay GPU_MT_TABLE Base.@assume_effects :foldable :native_executable function fact(x::Int)
    1 < x < 20 || raise_on_gpu("x is too big")
    return factorial(x)
end
raise_on_gpu(x) = #=do something with GPU=# error(x)

@Seelengrab
Copy link
Contributor

The :native_executable setting asserts that this method can be executed using Julia's native compiler and runtime.

Would it be more accurate to call this "host compiler & runtime"? It's possible to have a native runtime for the target architecture that's different from the runtime where the compilation happens.

@Keno
Copy link
Member

Keno commented Aug 28, 2023

I'm not sure the semantics of this effect override are precisely enough defined. I think we need to be very clear and careful what this is promising here. For the purposes of allowing constprop, I think all we need is that the overlay is consistent (in the :consistent sense) with the non-overlayed method. If that's what we want to do, then I think we should define it that way and choose a name that reflects it. In general, I want to be very careful to choose semantics that could in principle be checked. As currently presented, the word semantics in the definition is a bit nebulous.

@aviatesk aviatesk force-pushed the avi/nonoverlayed-effects branch from ad24c70 to 00b5060 Compare August 29, 2023 03:53
@aviatesk aviatesk force-pushed the avi/gpucompiler-384 branch from ec29009 to fe13b92 Compare August 29, 2023 03:54
Base automatically changed from avi/nonoverlayed-effects to master August 29, 2023 08:11
Certain external `AbstractInterpreters`, such as GPUCompiler.jl, have
long sought the ability to allow concrete evaluation for specific
overlay-ed methods to achieve optimal inference accuracy. This is
currently not permitted, although it should be safe when an overlay-ed
method has the same semantics as the original method, and its result can
be safely replaced with the result of the original method. Refer to
JuliaGPU/GPUCompiler.jl#384 for more examples.

To address this issue, this commit introduces the capability to override
the `:nonoverlayed` effect bit using `@assume_effects`. With the
enhancements in PR #51078, this override behaves similarly to other
effect bits. Consequently, external `AbstractInterpreters` can utilize
this feature to permit concrete evaluation for annotated overlay-ed
methods, e.g.
```julia
@overlay OVERLAY_MT Base.@assume_effects :nonoverlayed f(x) = [...]
```

However, it now seems awkward to annotate a method with `Base.@assume_effects :nonoverlayed`
when it is actually marked with `@overlay`. A more intuitive terminology,
like `native_executable`, might be more appropriate for renaming the
`:nonoverlayed` effect bit.
@aviatesk
Copy link
Member Author

This PR has gotten quite stale, so I'm going to close it for now. I'll have a new one ready soon, and we can pick up the discussion on that one.

@aviatesk aviatesk closed this May 1, 2024
@aviatesk aviatesk deleted the avi/gpucompiler-384 branch May 1, 2024 06:57
aviatesk added a commit that referenced this pull request Jun 18, 2024
This PR serves to replace #51080 and close #52940.
It extends the `:nonoverlayed` to `UInt8` and introduces the
`CONSISTENT_OVERLAY` effect bit, allowing for concrete evaluation of
overlay methods using the original non-overlayed counterparts when
applied. This newly added `:nonoverlayed`-bit is enabled through the
newly added `Base.Experimental.@consistent_overlay mt def` macro.
`@consistent_overlay` is similar to `@overlay`, but it sets the
`:nonoverlayed`-bit to `CONSISTENT_OVERLAY` for the target method
definition, allowing the method to be concrete-evaluated.
To use this feature safely, I have also added quite precise 
documentation to `@consistent_overlay`.
aviatesk added a commit that referenced this pull request Jun 18, 2024
This PR serves to replace #51080 and close #52940.
It extends the `:nonoverlayed` to `UInt8` and introduces the
`CONSISTENT_OVERLAY` effect bit, allowing for concrete evaluation of
overlay methods using the original non-overlayed counterparts when
applied. This newly added `:nonoverlayed`-bit is enabled through the
newly added `Base.Experimental.@consistent_overlay mt def` macro.
`@consistent_overlay` is similar to `@overlay`, but it sets the
`:nonoverlayed`-bit to `CONSISTENT_OVERLAY` for the target method
definition, allowing the method to be concrete-evaluated.
To use this feature safely, I have also added quite precise
documentation to `@consistent_overlay`.
aviatesk added a commit that referenced this pull request Jun 18, 2024
This PR serves to replace #51080 and close #52940.
It extends the `:nonoverlayed` to `UInt8` and introduces the
`CONSISTENT_OVERLAY` effect bit, allowing for concrete evaluation of
overlay methods using the original non-overlayed counterparts when
applied. This newly added `:nonoverlayed`-bit is enabled through the
newly added `Base.Experimental.@consistent_overlay mt def` macro.
`@consistent_overlay` is similar to `@overlay`, but it sets the
`:nonoverlayed`-bit to `CONSISTENT_OVERLAY` for the target method
definition, allowing the method to be concrete-evaluated.
To use this feature safely, I have also added quite precise
documentation to `@consistent_overlay`.
KristofferC pushed a commit that referenced this pull request Jun 18, 2024
This PR serves to replace #51080 and close #52940.
It extends the `:nonoverlayed` to `UInt8` and introduces the
`CONSISTENT_OVERLAY` effect bit, allowing for concrete evaluation of
overlay methods using the original non-overlayed counterparts when
applied. This newly added `:nonoverlayed`-bit is enabled through the
newly added `Base.Experimental.@consistent_overlay mt def` macro.
`@consistent_overlay` is similar to `@overlay`, but it sets the
`:nonoverlayed`-bit to `CONSISTENT_OVERLAY` for the target method
definition, allowing the method to be concrete-evaluated.
To use this feature safely, I have also added quite precise
documentation to `@consistent_overlay`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:effects effect analysis
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants