Added Mish Activation Function

[digantamisra98]

Resolves 168 and adds Mish Activation Function.

• Implementation of activation class named as MishActivation
• Documentation of activation function
• JUnit tests in api module
• Support for export of activation function to JSON (see ModelSaver.kt)
• Support for import of activation function from JSON (see ModelLoader.kt)

[digantamisra98]

@zaleslaw This is ready for review.

[zaleslaw]

Many thanks for the contribution, @digantamisra98 I'll run the TC and merge it if it is successful

[zaleslaw]

; -> , due to enum in Kotlin https://kotlinlang.org/docs/enum-classes.html

[zaleslaw]

Did you compile the Kotlin code or just added rows in the Notepad++? Because the code could not be compiled on TC servers

  > Task :api:compileKotlin
13:45:25   :api:compileKotlin
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/activation/Activations.kt: (234, 5): Expecting member declaration
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/activation/Activations.kt: (234, 5): Expecting member declaration
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/activation/Activations.kt: (258, 17): Unresolved reference: Mish
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/activation/Activations.kt: (258, 17): Unresolved reference: Mish
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelLoader.kt: (430, 9): Unresolved reference: ACTIVATION_MISH
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelLoader.kt: (430, 40): Unresolved reference: Mish
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelSaver.kt: (270, 21): Unresolved reference: Mish
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelLoader.kt: (430, 9): Unresolved reference: ACTIVATION_MISH
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelSaver.kt: (270, 29): Unresolved reference: ACTIVATION_MISH
13:45:47     
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelLoader.kt: (430, 40): Unresolved reference: Mish
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelSaver.kt: (270, 21): Unresolved reference: Mish
13:45:47     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelSaver.kt: (270, 29): Unresolved reference: ACTIVATION_MISH
13:45:47   > Task :api:compileKotlin FAILED

[zaleslaw]

:api:compileKotlin
18:15:27     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelLoader.kt: (430, 9): Unresolved reference: ACTIVATION_MISH
18:15:27     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelSaver.kt: (270, 29): Unresolved reference: ACTIVATION_MISH
18:15:27     
18:15:27     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelLoader.kt: (430, 9): Unresolved reference: ACTIVATION_MISH
18:15:27     e: /mnt/agent/work/c1e3495e9fda7093/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/inference/keras/ModelSaver.kt: (270, 29): Unresolved reference: ACTIVATION_MISH

Not compiled again, ok, I'll help to finish this request, clone your branch tomorrow and resolve the issue.

[mkaze]

@digantamisra98 Just add the following line to the KerasConstants.kt file:

internal const val ACTIVATION_MISH: String = "mish"

[digantamisra98]

@mkaze Done, thanks. Had missed that out.

@zaleslaw Fixed the issues and yes I had just edited/ added the files in an IDE, I didn't build it locally. I hope it compiles now.

[zaleslaw]

Thanks, @digantamisra98, for the contribution and @mkaze for the advice!
Good luck with ML science, and we will wait for new papers and results which could be implemented in DL frameworks and useful for the AutoML!