Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Mish activation function #168

Closed
5 tasks
zaleslaw opened this issue Jul 29, 2021 · 4 comments · Fixed by #173
Closed
5 tasks

Add Mish activation function #168

zaleslaw opened this issue Jul 29, 2021 · 4 comments · Fixed by #173
Assignees
Labels
good first issue Good for newcomers
Milestone

Comments

@zaleslaw
Copy link
Collaborator

The Mish activation function is missed in KotlinDL.

Read more about Mish: A Self Regularized Non-Monotonic Neural Activation Function

The desired PR addressing this issue should include:

  • Implementation of activation class named as MishActivation (you can take inspiration from the implementation of HardSigmoid as reference) added to the Activations.kt file
  • Documentation of activation function
  • JUnit tests in api module
  • Support for export of activation function to JSON (see ModelSaver.kt)
  • Support for import of activation function from JSON (see ModelLoader.kt)

P.S. If you want to take this ticket, please leave the comment below
P.P.S Read the Contributing Guidelines.

The reference implementation could be taken from tensorflow-addons

@zaleslaw zaleslaw added the good first issue Good for newcomers label Jul 29, 2021
@zaleslaw zaleslaw added this to the 0.3 milestone Jul 29, 2021
@zaleslaw
Copy link
Collaborator Author

Hi, @digantamisra98! Please leave a comment here to assign this ticket to you.
As I understand, you are the author of the Mish paper, a really great paper and results!

@digantamisra98
Copy link
Contributor

@zaleslaw Yes, I am taking this up and will be happy to add others over the weekend (hopefully). Thank you for your compliment and willingness to add it in the library :)

@zaleslaw
Copy link
Collaborator Author

zaleslaw commented Jul 30, 2021

@digantamisra98 I hope that KotlinDL could be more open to DL innovations than Keras with huge codebase and be faster in implementation of new ideas, will wait review request from you.

@digantamisra98
Copy link
Contributor

The PR is ready for review (I don't know if I can like request a review on the PR itself unless I'm a collaborator with write access).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants