--Ojanewton breaks with -b 24

[arielf]

In the vw source tree:

# takes about 0.25 sec, average loss = 0.089070
vw  --OjaNewton --sketch_size=10 --alpha_inverse=1.0 -d test/train-sets/0002.dat

Same command with -b 24:

# Takes over 16 seconds, has NaNs in all examples & error jumps to 0.283062
vw  -b 24 --OjaNewton --sketch_size=10 --alpha_inverse=1.0 -d test/train-sets/0002.dat

[arielf]

Also noticed that -b 20 still works reasonably well (although slower than default) and the breaking point is -b 21.

[JohnLangford]

I'm not sure what's wrong here. A careful examination of the computation is required. @haipengl do you want to take a look?

[yannstad]

The independent sample r1 in the box-muller transform must be strictly positive. But it is chosen using merand48, that possibly returns 0:

vowpal_wabbit/vowpalwabbit/OjaNewton.cc

Line 82 in 6ef7676

float r1 = merand48(all->random_state);