You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Using quantile to compute the median also returns an answer that is not correct:
julia>quantile(Int8[-128, 127], .5)
-128.5
The equivalent function in NumPy, np.quantile, returns the correct result in all cases:
julia>using PyCall
julia> np =pyimport("numpy");
julia> a = np.array(Int8[-128, 127], dtype=np.int8);
julia> np.quantile(a, 1)
127
julia>functiontest_python()
quantiles = (1, 0.5)
types = (
Int8 => np.int8,
Int16 => np.int16,
Int32 => np.int32
)
for q in quantiles, (type, dtype) in types
A = np.array([typemin(type), typemax(type)]; dtype)
result = np.quantile(A, q)
correct_result =quantile(map(Int, A), q)
println((; q, type, result, correct_result))
endend;
julia>test_python()
(q =1, type = Int8, result =127, correct_result =127)
(q =1, type = Int16, result =32767, correct_result =32767)
(q =1, type = Int32, result =2147483647, correct_result =2147483647)
(q =0.5, type = Int8, result =-0.5, correct_result =-0.5)
(q =0.5, type = Int16, result =-0.5, correct_result =-0.5)
(q =0.5, type = Int32, result =-0.5, correct_result =-0.5)
The text was updated successfully, but these errors were encountered:
yurivish
changed the title
The quantile function can return incorrect results for some integer arrays (Int8, Int16, Int32)
The quantile function can return incorrect results for integer arrays (Int8, Int16, Int32)
Jun 27, 2022
For example, using
quantile
to compute the maximum of the array[-128, 127]
incorrectly returns-129
if the array consists ofInt8
values:This happens because of the following code in
_quantile
:Statistics.jl/src/Statistics.jl
Line 1010 in 0588f2c
For the above array, that line computes the following expression, causing the incorrect result:
This behavior occurs with arrays of
Int8
,Int16
, andInt32
:Using
quantile
to compute the median also returns an answer that is not correct:The equivalent function in NumPy, np.quantile, returns the correct result in all cases:
The text was updated successfully, but these errors were encountered: