Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: break up monolithic sysimg build work #38119

Closed
wants to merge 1 commit into from

Conversation

vtjnash
Copy link
Member

@vtjnash vtjnash commented Oct 20, 2020

This alters the way that we build the system image, in preparation for future work in this direction. We've been talking about having better stacked and incremental system images for some time now. While this doesn't yet aim to fully accomplish that, I believe this does functionally simulate it quite well. Along the way, this lets us start identifying and fixing mistaken code where some piece accidentally depended on another piece, without declaring that dependency. It may also be seen as a template for PackageCompiler developers, as they should now better recognize some of the code involved here (particularly stdlib/generate_stdlib.jl) as being already in use. The general compile policies can now be seen to be split here between compile processes (such as Base.jl), linker processes (generate_minimal.jl and generate_full.jl), and their driver scripts (generate_sysimg.jl and generate_precompile.jl).

Implementation wise, this currently works a bit of slight-of-hand and knows that the default image "usr/lib/sys.so" has a better copy "usr/lib/fullsys.so" and will secretly open that one instead when you're using Julia interactively (in the future, we want this to be a bit smoother by delaying the decision until later). But otherwise (i.e. when you're using it non-interactively), it will default to opening the more minimal version, and thus be able to develop and replace the extra stdlibs. The results should be cross-compatible however, so the precompile caches for julia-{minimal|full}-{debug|release} can all be shared.

Opening as draft, since I'm running into this Pkg bug when attempting doctests: JuliaLang/Pkg.jl#2146 and this one too #38116
This also includes #37844, because those sort of introspection-based hacks to bypass correct Project.toml dependency usage are now intentionally not possible.

@vtjnash vtjnash added this to the 1.6 features milestone Oct 20, 2020

# Set up Main module
using .Base
import Base.MainInclude: eval, include
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be really nice if this can somehow be done elsewhere. The idea was to make Base more like a normal package; it should only create the Base module.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can go elsewhere, but I thought it felt the most natural here (it means that if you load just -J Base.so, it acts just like you are just inside a fresh module Main, instead of being a baremodule Main).

@vtjnash vtjnash force-pushed the jn/build-incremental branch from dad9fbb to 61ee58a Compare November 23, 2020 23:31
@vtjnash vtjnash mentioned this pull request Dec 1, 2020
3 tasks
@vtjnash vtjnash closed this May 25, 2023
@vtjnash vtjnash deleted the jn/build-incremental branch May 30, 2023 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants