Use SnoopCompileBot #2832

daschw · 2020-06-30T18:34:46Z

This should automatically update the precompile files using SnoopCompileBot and a corresponding Github Action as described in https://timholy.github.io/SnoopCompile.jl/stable/bot/
This removes the current precompile file and hopefully PRs with new precompile files are opened after this gets merged.
@timholy or @aminya it would be great if you could take a look at this and check if I'm doing something wrong here.

daschw · 2020-06-30T19:02:18Z

The SnoopCompile runs can be checked at https://github.com/daschw/Plots.jl/actions/runs/153066245 and here you can take a look at the PR that would be created: daschw#4

daschw · 2020-06-30T19:18:31Z

I guess we should decide if we want to have different precompile files per OS (does this make sense?). Otherwise we don't have to run the action on all OSs. And, since nightly is already 1.6 I should add a run for some Julia 1.5 version

timholy · 2020-06-30T19:19:25Z

LGTM but when it comes to the bot, @aminya is the real expert.

timholy · 2020-06-30T19:28:05Z

Also worth saying, I think you guys will see some gains wrt to load times within another month or so on nightly...there are still several crucial PRs to merge, the most important being JuliaLang/julia#35877 (which is currently on hold for good reasons). Outside of Julia & Pkg, only JuliaWeb/HTTP.jl#549 is waiting to be merged (as long as you're using at least FixedPointNumbers 0.8.1, ColorTypes 0.10.4). For me the combination of all of those drops TTFP by almost an additional factor of 1.5-2 compared to 1.5rc1.

EDIT: these changes should also make the precompile files more effective, which is why I brought it up here. Currently you can "ask" for something to be precompiled but it doesn't always "take," if any MethodInstances it depends on got invalidated. So stomping out invalidations reduces the number of failures to precompile. There are other reasons to fail but this is the biggest one in practice.

daschw · 2020-06-30T19:36:34Z

Sounds great and huge thanks to everyone who's involved in this!

daschw · 2020-06-30T20:15:33Z

Looking at https://github.com/daschw/Plots.jl/pull/4/files now it seems like that the precompile files are identical for each OS, so probably we do not require multi-os.

deps/SnoopCompile/snoop_bench.jl

deps/SnoopCompile/snoop_bot.jl

Co-authored-by: Amin Yahyaabadi <aminyahyaabadi74@gmail.com>

aminya · 2020-06-30T21:22:52Z

.github/workflows/SnoopCompile.yml

+        version:   # NOTE: the versions below should match those in your botconfig
+          - '1.3'
+          - '1.4'
+          # - '1.5'


Enabling this since nightly is 1.6 now? 🤔

Suggested change

# - '1.5'

- '1.5.0-rc1'

I tried but it did not work: https://github.com/daschw/Plots.jl/actions/runs/153176644

All 1.5 tests are failing there.

Maybe try -'1.5.0-rc1'.

@SaschaMann shouldn't 1.5 install '1.5.0-rc1' by default? julia-setup fails to do this.

ok, let's try

With 1.5.0-0 I get

Run julia-actions/setup-julia@latest 0s PLOTS_TEST: true Run julia-actions/setup-julia@latest with: version: 1.5.0-0 arch: x64 show-versioninfo: false env: GKS_ENCODING: utf8 GKSwstype: 100 PLOTS_TEST: true ##[error]Unexpected HTTP response: 404

https://github.com/daschw/Plots.jl/runs/824362165

My bad, it should've been ^1.5.0-0. Sorry! 🤦

That does not work either: https://github.com/daschw/Plots.jl/actions/runs/155304982

But that's an issue with SnoopCompile.jl cc: @aminya

Co-authored-by: Amin Yahyaabadi <aminyahyaabadi74@gmail.com>

daschw · 2020-06-30T21:38:42Z

@aminya I guess, once we're sure this works I should uncomment

    #  - 'master'  # NOTE: to run the bot only on pushes to master

in SnoopCompile.yml?

.github/workflows/SnoopCompile.yml

aminya · 2020-06-30T21:46:01Z

@aminya I guess, once we're sure this works I should uncomment
    #  - 'master'  # NOTE: to run the bot only on pushes to master
in SnoopCompile.yml?

If you want to get precompile updates on pushes to other branches (under JuliaPlots/Plots or when users fork this repository) don't uncomment this. But if you want precompile updates only on pushes to master, then do that.

I don't think this be an issue but may happen in rare cases: Once this PR is merged and bot PR is merged as well, it also may worth mentioning in the contribution section of readme or docs to set should_precompile=false in case some error happens because of precompilation. This is very rare, but it is good to know.

daschw · 2020-06-30T21:47:58Z

it also may worth mentioning in the contribution section of readme or docs to set should_precompile=false

Where exactly do we set that?

aminya · 2020-06-30T21:49:18Z

it also may worth mentioning in the contribution section of readme or docs to set should_precompile=false

Where exactly do we set that?

https://github.com/daschw/Plots.jl/pull/4/files#diff-2270ce88ed8290c3b808f5ea76728946R1
Again, this is very rare, and for Plots, this will not be a thing as it has used hardcoded precompile signatures for a long time.

daschw · 2020-07-01T07:16:47Z

OK, this seems to work now. Thanks a lot @aminya @timholy and @SaschaMann for your support!

daschw added 12 commits June 30, 2020 18:28

automate updating of precompilation files using SnoopCompileBot

12df81f

don't run snoopcompilebot on nightly

3dd3808

remove nightly in snoop_bot.jl

ddcf443

don't include precompile_loader

fb402d9

xvfb-run and update versions

86b9269

string instead of versionnumber

acd6cfa

add GKS_ENCODING env variable

d0920ac

actually use xvfb-run

2511d21

add env variables for GR

3ea5217

skip statsplots testimages

cfab4e3

don't display plotly plots

4f791a3

be explicit about julia versions

1661843

multios and run on 1.5

3f521b3

don't run on 1.5

a2aad6b

aminya reviewed Jun 30, 2020

View reviewed changes

deps/SnoopCompile/snoop_bench.jl Outdated Show resolved Hide resolved

aminya reviewed Jun 30, 2020

View reviewed changes

deps/SnoopCompile/snoop_bot.jl Outdated Show resolved Hide resolved

daschw and others added 2 commits June 30, 2020 23:22

Update deps/SnoopCompile/snoop_bench.jl

032f76c

Co-authored-by: Amin Yahyaabadi <aminyahyaabadi74@gmail.com>

Update deps/SnoopCompile/snoop_bot.jl

6425ff8

Co-authored-by: Amin Yahyaabadi <aminyahyaabadi74@gmail.com>

aminya reviewed Jun 30, 2020

View reviewed changes

Update .github/workflows/SnoopCompile.yml

fc89578

Co-authored-by: Amin Yahyaabadi <aminyahyaabadi74@gmail.com>

daschw commented Jun 30, 2020

View reviewed changes

.github/workflows/SnoopCompile.yml Outdated Show resolved Hide resolved

Update .github/workflows/SnoopCompile.yml

dc81c6a

daschw added 2 commits July 1, 2020 00:06

update 1.5 version specification in SnoopCompile.yml

a46ffbe

use '1.5.0-rc1' until 1.5 is released

79fac7a

daschw merged commit 065089a into JuliaPlots:master Jul 1, 2020

daschw deleted the snoopibot branch October 12, 2020 09:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use SnoopCompileBot #2832

Use SnoopCompileBot #2832

daschw commented Jun 30, 2020

daschw commented Jun 30, 2020

daschw commented Jun 30, 2020

timholy commented Jun 30, 2020

timholy commented Jun 30, 2020 •

edited

Loading

daschw commented Jun 30, 2020

daschw commented Jun 30, 2020 •

edited

Loading

aminya Jun 30, 2020 •

edited

Loading

daschw Jun 30, 2020

daschw Jun 30, 2020

aminya Jun 30, 2020

daschw Jun 30, 2020

daschw Jun 30, 2020

daschw Jun 30, 2020

SaschaMann Jul 1, 2020

daschw Jul 2, 2020

daschw Jul 2, 2020

daschw commented Jun 30, 2020

aminya commented Jun 30, 2020 •

edited

Loading

daschw commented Jun 30, 2020

aminya commented Jun 30, 2020

daschw commented Jul 1, 2020

Use SnoopCompileBot #2832

Use SnoopCompileBot #2832

Conversation

daschw commented Jun 30, 2020

daschw commented Jun 30, 2020

daschw commented Jun 30, 2020

timholy commented Jun 30, 2020

timholy commented Jun 30, 2020 • edited Loading

daschw commented Jun 30, 2020

daschw commented Jun 30, 2020 • edited Loading

aminya Jun 30, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daschw commented Jun 30, 2020

aminya commented Jun 30, 2020 • edited Loading

daschw commented Jun 30, 2020

aminya commented Jun 30, 2020

daschw commented Jul 1, 2020

timholy commented Jun 30, 2020 •

edited

Loading

daschw commented Jun 30, 2020 •

edited

Loading

aminya Jun 30, 2020 •

edited

Loading

aminya commented Jun 30, 2020 •

edited

Loading