Use a distinct interactive system image, isys.so, if Julia is interactive. Fallback to sys.so.

[mkitti]

If Julia is running interactively, attempt to load isys.so rather than sys.so as the system image. If isys.so is not found, fallback to using sys.so. This allows the user or perhaps a package to install a distinct system image for interactive use.

This pull request arose as a proof of concept from a Discourse discussion.

The idea is that the user may want a system image that includes REPL.jl, Pkg.jl, and other interactive packages (e.g. OhMyREPL.jl) when using Julia interactively due to faster load times. However, having these in the system image is not necessary for non-interactive use. The REPL.jl and Pkg.jl packages were removed from the standard system image in Julia 1.11.

The following new C API functions are added.

• jl_is_interactive
• jl_get_interactive_sysimg_path
• jl_preload_successful

Demonstration

Here's a demonstration of how this currently works.

Initially, isys.so does not exist, so Julia will load sys.so.

~/src/julia$ ./julia --banner=no
julia> sysimg = Base.JLOptions().image_file |> unsafe_string
"/home/mkitti/src/julia/usr/lib/julia/sys.so"

julia> cp(sysimg, replace(sysimg, "sys.so" => "isys.so"))
"/home/mkitti/src/julia/usr/lib/julia/isys.so"

julia> exit()

We copied sys.so to isys.so. Now isys.so exists, so we shoud load it in interactive mode.

~/src/julia$ ./julia --banner=no
julia> sysimg = Base.JLOptions().image_file |> unsafe_string
"/home/mkitti/src/julia/usr/lib/julia/isys.so"

If we are not running in interactive mode, then the regular sys.so is loaded despite isys.so existing.

~/src/julia$ ./julia -e "Base.JLOptions().image_file |> unsafe_string |> println"
/home/mkitti/src/julia/usr/lib/julia/sys.so

With these changes, the user can create an isys.so that is optimized for interactive use while keeping a separate sys.so for non-interactive use.

Questions

1. Where should the interactive system image be located? If this is meant to be controlled by the user, perhaps ~/.julia/sysimgs/ would be a good alternative place.
2. Should we use jl_is_interactive to set the global variable Base.is_interactive or implement Base.isinteractive()?

[vchuravy]

I don't think we can do this. Besides the extra storage cost, standard library packages will become like normal Julia package and thus these additional syringes will become stale and out of date, adding quite an additional bit of "What packages are being used" and " Why does this not have the new feature" complexity to the user.

Edit: I see that this PR doesn't yet propose that we ship an isys.so, but I will point out that every system image bifurcates the cache. So you will have to precompile everything twice.

[vtjnash]

Duplicate of #38119, which used to avoid that caching problem, but for various reasons cannot anymore without some significant added difficulty. Happy to discuss why we closed that and answer questions though

[mkitti]

As Valentin noticed, this pull request is relatively simple. It does not build a second system image. A second interactive system image is left to the user to obtain or build. The scope of this pull request does not include building a second system image.

This pull request merely looks for an isys.so if Julia is running interactively. If present it will use that. Otherwise, it will falllback to using sys.so.

The intention is that the building of an additonal system image be deferred to the packages. Perhaps we have a package that depends on REPL.jl and Pkg.jl and then rebuilds isys.so everytime those packages are updated. Alternatively, the package downloads an artifact containing a system image compiled elsewhere.

Thus, this pull request is much simpler than #38119. Furthermore, it requires the user to opt-in to the use of two system images and the bifurcated cache. It enables the user to switch system images based on the interactive state of Julia.

[mkitti]

A reduced version of this pull request would focus on exposing the interactive state via jl_is_interactive. Julia could then be embedded into another CLI driver which implements the system image switching behavior.

[PallHaraldsson]

Isn't the case for Pkg simply handled by starting Pkg in your startup.jl file conditionally if interactive? And REPL.jl much less of an issue. Could installing Julia provide such a default startup.jl file?

[I'm not sure the startup is much of an issue unless on really slow computers, e.g. Raspberry Pi.]

[mkitti]

Would the C routine jl_is_interactive be of interest so that a CLI or other customized Julia driver might be able to read the calculated is_interactive state without first loading Julia?

Otherwise, the the is_interactive state is not determined until later in client.jl.

julia/base/client.jl

Line 272 in e9b0fa1

is_interactive::Bool |= interactiveinput