Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long precompilation times #42

Open
mwallerb opened this issue Jul 22, 2023 · 6 comments
Open

Long precompilation times #42

mwallerb opened this issue Jul 22, 2023 · 6 comments

Comments

@mwallerb
Copy link
Collaborator

SparseIR is now super-fast to load. However, it does take quite long to precompile.

I don't understand why this should be, since we mostly do bog-standard linear algebra ... or is the quad-precision linear algebra just much slower?

Quite a small issue, but it is a little annoying :)

@shinaoka
Copy link
Member

On my Macbook Pro M1

julia> @time Base.compilecache(Base.identify_package("SparseIR"))
[ Info: Precompiling SparseIR [4fe2279e-80f0-4adb-8463-ee114ff56b7d]
 20.228946 seconds (3.60 k allocations: 326.414 KiB)

Yes, it is a bit long, although I was not aware of this issue.

@SamuelBadr
Copy link
Collaborator

Disabling quad-precision (i.e. replacing Float64x2 by Float64) reduces precompilation from ~45 to ~30 seconds on my laptop. I have tried in the past to speed this up, but don't really have a clue how to diagnose where the time is spent.

@terasakisatoshi
Copy link
Contributor

As of Julia 1.9, Julia introduced Package images.

https://docs.julialang.org/en/v1/devdocs/pkgimg/#pkgimages.

Package images are shared libraries that contain both code and data. Like .ji cache files, they are generated per package.

This feature lets us quickly load SparseIR.jl (i.e., using SparseIR). However, it increases the time for precompilation.

image

@terasakisatoshi
Copy link
Contributor

Maybe using julia --pkgimage=no solves our issue.

image

@shinaoka
Copy link
Member

shinaoka commented Dec 12, 2024

But, in the end, we will switch the backend of SparseIR.jl to C++/C. The precompilation time will be reduced then.

@SamuelBadr
Copy link
Collaborator

In 1.12, there are tools to profile precompilation https://docs.julialang.org/en/v1.12-dev/manual/performance-tips/#Reducing-precompilation-time. Haven't yet tried it though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants