-
Notifications
You must be signed in to change notification settings - Fork 421
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix sampling from distributions with integer-valued parameters (e.g. MvNormal
and Dirichlet
)
#1262
Conversation
I think this one is a bit different than the In any case, I don't think this PR is the right fix since it "lies" about the element type. The element type usually refers to the type of the parameters and not the type of the sample space. I think the simplest fix would be to promote to floats in Distributions.jl/src/multivariate/dirichlet.jl Lines 28 to 35 in 0da0bcb
Distributions.jl/src/multivariates.jl Line 77 in 0da0bcb
Float64 and Int respectively. I believe the former is the less controversial solution.
|
Thanks for your comment! I think you touch on different issues, I'll try to respond as briefly as possible.
IMO the natural approach is to let users specify the type of the samples of the underlying RNG, which would be A while ago I mentioned this view also in a discussion about MeasureTheory in which AFAIK the support of a measure is encoded in its type. I think it can be difficult to know it in advance and could be avoided by specifying the type of the most basic sampling calls.
I think it really boils down to the question what exactly If it should be the type of the parameters (which is e.g. not enforced in the type of I am aware of the many issues and discussions in Distributions regarding
This would be a simple fix but the allocations seem wasteful and only necessary to fix the incorrect heuristic in
I think this will break many use cases with non-standard parameter types that are handled by the current heuristic. Taken together, I think the most natural and possibly appropriate fix would be to change the heuristic of the type of the random variates in the default implementation of |
I'm not that worried about the extra allocation. If it matters for a user then it would be easy for a user to change their code to construct the parameter vector with floating-point elements.
I'd be surprised if that is really the case since you can't really rely on such behavior for There is the problem with pretending that the variates follow the julia> typeof(rand(Gamma(0.5f0)))
Float64
julia> typeof(rand(Gamma(0.5)))
Float64
julia> typeof(rand(Gamma(big(0.5))))
BigFloat but all these numbers are based on the Distributions.jl/src/samplers/gamma.jl Line 180 in 01ba56b
|
If Distributions.jl/src/multivariates.jl Lines 76 to 77 in 01ba56b
An example that would be broken by hardcoding julia> using Distributions, ForwardDiff
julia> s = MvNormal(ForwardDiff.Dual.(ones(10)));
julia> rand(s)
10-element Array{ForwardDiff.Dual{Nothing,Float64,0},1}:
Dual{Nothing}(0.1666028548436855)
Dual{Nothing}(-0.43914145214951167)
Dual{Nothing}(-1.0691807830172526)
Dual{Nothing}(1.8454427275951966)
Dual{Nothing}(0.39196945140339634)
Dual{Nothing}(-1.046448411408746)
Dual{Nothing}(-0.7625542381972902)
Dual{Nothing}(0.5384999518634571)
Dual{Nothing}(-1.266646900649526)
Dual{Nothing}(-0.37844015490701083)
julia> eltype(s)
ForwardDiff.Dual{Nothing,Float64,0} Basically, anything where Instead I would suggest adding rand(rng::AbstractRNG, s::Sampleable{Multivariate,Continuous}) =
_rand!(rng, s, Vector{float(eltype(s))}(undef, length(s))) (and similarly for other
I'm sorry, isn't this exactly what currently Distributions assumes in the implementation of
Sorry, I am not sure what these comments refer to. Distributions.jl/src/samplers/gamma.jl Line 180 in 01ba56b
is exactly one of these "base methods" that I referred to above. My suggestion was to allow to specify the type of exactly (and only) these base variates, i.e., to change such calls to x = randn(rng, T) where julia> typeof(rand(Float32, Gamma(0.5f0)))
Float32
julia> typeof(rand(Float32, Gamma(0.5)))
Float64
julia> typeof(rand(Float32, Gamma(big(0.5))))
BigFloat
julia> typeof(rand(BigFloat, Gamma(0.5f0))) # requires https://github.com/JuliaLang/julia/pull/35111
BigFloat
julia> typeof(rand(BigFloat, Gamma(0.5))) # requires https://github.com/JuliaLang/julia/pull/35111
BigFloat
julia> typeof(rand(BigFloat, Gamma(big(0.5)))) # requires https://github.com/JuliaLang/julia/pull/35111
BigFloat But it seems this discussion is a bit too general here and not needed to fix the problem in the OP. Coming back to the PR, would it be OK to revert the rand(rng::AbstractRNG, s::Sampleable{Multivariate,Continuous}) =
_rand!(rng, s, Vector{float(eltype(s))}(undef, length(s))) ? |
BTW I just checked, the suggested change of julia> d = MvNormal([0, 0], [1, 1]);
julia> rand(d)
ERROR: MethodError: no method matching randn(::Random._GLOBAL_RNG, ::Type{Int64})
Closest candidates are:
randn(::Random.AbstractRNG, ::Type{T}, ::Tuple{Vararg{Int64,N}} where N) where T at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Random/src/normal.jl:201
randn(::Random.AbstractRNG, ::Type{T}, ::Integer, ::Integer...) where T at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Random/src/normal.jl:204
randn(::Random.AbstractRNG) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Random/src/normal.jl:38
...
Stacktrace:
[1] randn!(::Random._GLOBAL_RNG, ::Array{Int64,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Random/src/normal.jl:178
[2] _rand! at /home/david/.julia/packages/Distributions/7Ifyw/src/multivariate/mvnormal.jl:275 [inlined]
[3] rand at /home/david/.julia/packages/Distributions/7Ifyw/src/multivariates.jl:76 [inlined]
[4] rand(::MvNormal{Int64,PDMats.PDiagMat{Int64,Array{Int64,1}},Array{Int64,1}}) at /home/david/.julia/packages/Distributions/7Ifyw/src/genericrand.jl:22
[5] top-level scope at REPL[5]:1 whereas with my suggestion julia> d = MvNormal([0, 0], [1, 1]);
julia> rand(d)
2-element Array{Float64,1}:
1.6521247327111197
-0.2234559539354925 |
With these issues, it seems that there isn't a single best solution. There are trade-offs and most of the options have aspects that are annoying. So I'm trying to make sure that we consider some of the implications before applying a fix. A drawback with one solution could be the introduction of allocations and the drawback with another solution could be that it follows a path that pretends that random number generation follows the precision of the parameters. A third solution might make it impossible to create
x = randn(rng, T) Originally, I thought you were suggesting to use It wouldn't help with the issue mentioned in my previous post where you could end up with Anyway, to avoid that the perfect is the enemy of the good here, I guess we should go with your latest proposal. |
Codecov Report
@@ Coverage Diff @@
## master #1262 +/- ##
==========================================
- Coverage 81.78% 81.23% -0.55%
==========================================
Files 117 115 -2
Lines 6565 6533 -32
==========================================
- Hits 5369 5307 -62
- Misses 1196 1226 +30
Continue to review full report at Codecov.
|
I applied the suggestion and added some tests for the uni- and multivariate cases (they revealed that currently |
Yes, I assume ideally internally one would promote the user-specified precision based on the involved parameters. |
Dirichlet
MvNormal
and Dirichlet
)
Thank you for merging the PR! Is it possible to make a new release with the fix? I already bumped the version in this PR 🙁 |
This PR fixes and tests sampling from
Dirichlet
with integer-valued parameters which is broken in 0.24.11 and not tested currently:IMO the main underlying problem is that
rand
falls back to the in-place methodrand!
and uses a heuristic to determine the type of the samples. This problem seems similar to the one described (and fixed) for(log)pdf
in #1257. However, I did not want to make any major to Distributions in this PR and therefore it is just a fix for the problem withDirichlet
.Edit: This PR also fixes #1004 and similar issues for other distributions by using containers with floating point numbers for sampling from continuous distributions in the default implementation of
rand
. E.g, the PR fixes the following example as well: