Precompilation does not produce a deterministic result

[timholy]

This is essentially a copy of JuliaPlots/Plots.jl#1379. I tested the precompilation of FileIO, commit fd7bf2f800d9119f2720a9054f7a319549eb2c8d with a using Nullables thrown in to make it build on

julia> versioninfo()
Julia Version 0.7.0-DEV.3693
Commit f7fe60b* (2018-02-02 13:33 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)
Environment:
  JULIAHOME = /home/tim/src/julia
  JULIAFUNCDIR = /home/tim/juliafunc
  JULIA_CPU_CORES = 2

Here are the results of multiple rounds of precompilation (I used using Example to make sure the code-loading machinery itself had been compiled):

tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no
julia> using Example

julia> @time using FileIO
  3.600703 seconds (3.65 M allocations: 191.672 MiB, 2.07% gc time)

julia> 
tim@diva:~/.julia/v0.7/FileIO/src$ ls -l ~/.julia/lib/v0.7/FileIO.ji 
-rw------- 1 tim holy 1897922 Feb  5 14:19 /home/tim/.julia/lib/v0.7/FileIO.ji
tim@diva:~/.julia/v0.7/FileIO/src$ touch FileIO.jl 
tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no -e 'using FileIO'
tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no
julia> using Example

julia> @time using FileIO
  2.297120 seconds (2.70 M allocations: 142.614 MiB, 2.93% gc time)

julia> 
tim@diva:~/.julia/v0.7/FileIO/src$ ls -l ~/.julia/lib/v0.7/FileIO.ji 
-rw------- 1 tim holy 1401622 Feb  5 14:21 /home/tim/.julia/lib/v0.7/FileIO.ji
tim@diva:~/.julia/v0.7/FileIO/src$ touch FileIO.jl 
tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no -e 'using FileIO'
tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no
julia> using Example

julia> @time using FileIO
  2.950153 seconds (2.53 M allocations: 136.272 MiB, 2.21% gc time)

julia> 
tim@diva:~/.julia/v0.7/FileIO/src$ ls -l ~/.julia/lib/v0.7/FileIO.ji 
-rw------- 1 tim holy 1904934 Feb  5 14:21 /home/tim/.julia/lib/v0.7/FileIO.ji
tim@diva:~/.julia/v0.7/FileIO/src$ touch FileIO.jl 
tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no -e 'using FileIO'
tim@diva:~/.julia/v0.7/FileIO/src$ julia-0.7 --startup-file=no -q --depwarn=no
julia> using Example

julia> @time using FileIO
  8.187372 seconds (7.58 M allocations: 401.505 MiB, 1.21% gc time)

julia> 
tim@diva:~/.julia/v0.7/FileIO/src$ ls -l ~/.julia/lib/v0.7/FileIO.ji 
-rw------- 1 tim holy 3261892 Feb  5 14:22 /home/tim/.julia/lib/v0.7/FileIO.ji

You can see that both the startup time and file sizes varied by more than a factor of 2, without any change to FileIO's code. Slower startup seems to be associated with bigger files.

[StefanKarpinski]

@vtjnash, any idea what's up here?

[tknopp]

This is the same as #24383 but for master and not 0.6.1/2, no?

[timholy]

Possibly, but I don't think that issue showed that the result of precompilation fluctuates wildly from run-to-run even when nothing changes in terms of the code itself (neither the code being compiled nor the julia version). #24383 might be a bug, or it might mean that we're using the wrong strategy; this issue clearly indicates a bug. Of course they might have the same root cause.

[tknopp]

#24383 is also JuliaIO/FileIO.jl#156 where fluctuations are shown. I see the same fluctuations on 0.6.2

[nico202]

Any way the non-determinism will be fixed? I tried precompiling in a (theoretically) clean environment
[script here https://gist.github.com/nico202/b9f32cee5f771c2a578c41e61d5ebd34] and every time the precompiled script changes (also in size!). Am I missing something (some global-state variable that must be kept fixed)?

versioninfo()                                                                                                                                                                             
Julia Version 1.0.0                                                                                                                                                                              
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
  OS: Linux (x86_64-unknown-linux-gnu)                                                                                                                                                           
  CPU: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz                                                                                                                                                  
  WORD_SIZE: 64                                                                                                                                                                                  
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)  ```

[nico202]

Is anybody looking at this? I find non-determinism scary

[nico202]

Progresses #34753 here!