-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use a stable rng from StableRNGs
for reproducibility
#170
Conversation
Codecov Report
@@ Coverage Diff @@
## master #170 +/- ##
==========================================
- Coverage 97.54% 97.47% -0.08%
==========================================
Files 109 109
Lines 6314 6332 +18
==========================================
+ Hits 6159 6172 +13
- Misses 155 160 +5 |
Thanks for this PR, I think StableRNG might indeed make sense for testing. But this PR would also add it outside of tests? I think we should avoid that, and only use StableRNG for the tests. We currently might still have some random function that take only a seed instead of a random generator. These functions should be refactored so that they use |
935063c
to
77a4e27
Compare
I've restored the
I've propagated a This took a lot of time / effort to fix all the tests, but upstream @simonschoelly, this PR is now ready for review / comments. |
I think I have addressed all the comments. Upstream tests ✔. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks a lot!
Amazing, thanks for your valuable comments (and obviously for your time !) 🎉 . |
Is is currently very hard (impossible ?) to rely on
Graphs
across different julia versions: this is observed inPlots.jl
ci through downstream test ofGraphRecipes.jl
.The reason is that the RNG across julia minors is documented to be unreliable in terms of reproducibility.
This makes
ci
testing via image comparison difficult unless duplicating reference files for different julia versions which is quite cumbersome.It is advised from julia docs, to use
StableRNGs.jl
instead.PR:
StableRNG
s in tests;rng
andseed
;update.ci
workflows