sigmoid_fast in GRU?

[oysteinsolheim]

Hi!
Is there any particular reason that sigmoid_fast is not used in the helper function _gru_output in the GRU layer in reccurent.jl? I see it used in the LSTM layer.

function _gru_output(gxs, ghs, bs)
  r = @. σ(gxs[1] + ghs[1] + bs[1])
  z = @. σ(gxs[2] + ghs[2] + bs[2])
  return r, z
end

In my (not so simple) use case i got a 5% speed up in the forward step using sigmoid_fast on cpu compared to the above version.

[ToucheSir]

If I had to guess, we just missed it. Care to file a PR?

[oysteinsolheim]

Sure!