Weight Initialisation might be a cause for loss NaN

[xiaoxuqi-ms]

As weights are initialized as codes line: 70 - line: 75, in file FFMWithAdag.scala, standard deviation of Z= W * X +b roughly equals to sqrt(mn/2). Once it is larger than 706(around), exp(z) becomes NaN. Proper initialization of weight should make the Guassion distribution more narrowed. coef = sqrt(1/mnk)?

[VinceShieh]

Thanks for raising this issue. Yes, proper initialization is quite important, but I dont get your point here. Note that, k is normally set to a small number.