Stabilize random number reproducibility in doctests #3472
Conversation
This is suggested by the Documenter documentation. I was not able to locally run make.jl until I did this. (Actually the situation was worse because I was running the tests with a different version of the package than the local one, which was yielding confusing error messages.)
|
This is what I ran from within the DataFrames directory which updated the docs Project.toml file and enabled me to run (DataFrames) pkg> activate docs
(docs) pkg> dev .I'm not sure if that relationship is fully captured by the code in the Project.toml file. |
|
Now the behavior flipped so this branch is failing and #3360 passed??? |
This was suggested in a discourse post where another package had random numbers change between Julia versions. Apparently this is expected behavior. https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/53
|
According to a recent discourse post, seeded random numbers can change between Julia versions. The StableRNGs.jl package was created for reproducible testing as used here, so I attempted to implement it. I'm not sure if all of the calls to |
nathanrboyer
changed the title
|
@nathanrboyer thank you for working on it. I think we also need an entry to Then let us merge this PR and merge #360 with |
|
I added a compat entry to the docs project.toml. I don't know of a way to do the same for test dependencies added as "extras"? |
| @@ -60,8 +60,21 @@ Logging = "56ddb016-857b-54e1-b83d-db4d58db5568" | |||
| OffsetArrays = "6fe1bfb0-de20-5000-8ca7-80f57d26f881" | |||
| ShiftedArrays = "1277b4bf-5013-50f5-be3d-901d8477a67a" | |||
| SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" | |||
| StableRNGs = "860ef19b-820b-49d6-a774-d7a799459cd3" | |||
There was a problem hiding this comment.
We need to also add these dependencies here as all extras entreis should have info in this file (we use them in tests not only in documentation building). Thank you!
|
Thank you! |
MersenneTwister(1234)from Random.jl yields different results now than those shown in the docstrings. I updated the docstrings ofshuffleandshuffle!so that their output matches the new results. I also had to modify thedocsProject.toml to be able to run the doctests correctly as suggested by Documenter.jl.This should fix the currently failing doctests in #3360.