Fix numerical precision problem with smax in hyperband

[KronosTheLate]

Motivation

For certain inputs, the flooring of smax, as prescribed in the original paper

, results in a lower smax than should be the case due to machine precision. This is particularly problematic because it is an issue for η = 3, which is the most common case. See the demonstration below:

julia> for max_reductions in 1:18
           η = 3
           R = η ^ max_reductions
           s_max = floor(Int, log(η, R))  # As perscribed in the paper
           @show max_reductions, log(η, R), s_max
       end
(max_reductions, log(η, R), s_max) = (1, 1.0, 1)
(max_reductions, log(η, R), s_max) = (2, 2.0, 2)
(max_reductions, log(η, R), s_max) = (3, 3.0, 3)
(max_reductions, log(η, R), s_max) = (4, 4.0, 4)
(max_reductions, log(η, R), s_max) = (5, 4.999999999999999, 4)
(max_reductions, log(η, R), s_max) = (6, 6.0, 6)
(max_reductions, log(η, R), s_max) = (7, 7.0, 7)
(max_reductions, log(η, R), s_max) = (8, 8.0, 8)
(max_reductions, log(η, R), s_max) = (9, 9.0, 9)
(max_reductions, log(η, R), s_max) = (10, 9.999999999999998, 9)
(max_reductions, log(η, R), s_max) = (11, 11.0, 11)
(max_reductions, log(η, R), s_max) = (12, 12.0, 12)
(max_reductions, log(η, R), s_max) = (13, 12.999999999999998, 12)
(max_reductions, log(η, R), s_max) = (14, 14.0, 14)
(max_reductions, log(η, R), s_max) = (15, 14.999999999999998, 14)
(max_reductions, log(η, R), s_max) = (16, 16.0, 16)
(max_reductions, log(η, R), s_max) = (17, 16.999999999999996, 16)
(max_reductions, log(η, R), s_max) = (18, 18.0, 18)

For some values, log(η, R) ends up being slightly lower than it is analytically. Combine this with a flooring operator, and you get problems - max_reductions is what you want, but s_max is what you get. For η ∈ {5, 6, 7, 8}, the output is occutionally slightly higher than it should be. This is no problem due to the floor operator, which is why the proposed fix only checks for equality when rounding up (by ceil), and fixes potential problems by rounding up. The undershooting is again a problem for η = 9, and that concludes my limited testing.

Proposed fix

To remedy this, I propose cheching to see if log(η, R) is an integer within the default precision, and if it is, rounding it. In my limited testing, I did not see this produce the wrong result at any point, but it does fix the problem.

[baggepinnen]

This looks like a nice thing to fix, thanks! There is a problem with the parenthesis in at least two of the expressions