Selu Operator not present in MXNet

[anirudhacharya]

Operator definition
y = gamma * (alpha * e^x - alpha) for x <= 0,
y = gamma * x for x > 0

@nswamy please label this - "Operator", "Feature Request"

[szha]

we have it in gluon. https://mxnet.incubator.apache.org/versions/master/api/python/gluon/nn.html?highlight=selu#mxnet.gluon.nn.SELU

[frankfliu]

Hi @anirudhacharya, thanks for submitting issue. @sandeep-krishnamurthy requesting this be labeled.

[gigasquid]

Thanks - It will be great to have selu available to the other language bindings too 👍

[Ldpe2G]

@gigasquid it is easy to implement selu in Scala:https://github.com/Ldpe2G/DeepLearningForFun/blob/master/Mxnet-Scala/SelfNormNets/src/main/scala/snns/Ops.scala

[gigasquid]

@Ldpe2G thanks for the reference. really nice 💯

[nswamy]

@Ldpe2G @gigasquid I am wondering if this should be implemented on the backend so all languages can benefit.

[gigasquid]

@nswamy Yes, I still think it would be beneficial to have it implemented in the backend. It would be best if users did not have to implement it themselves and could have it as a standard activation. I also appreciate the example from @Ldpe2G which is a nice reference for implementing operators, (especially using the where). I might port that into a Clojure example :)

[haojin2]

The fix for this issue is in #12059