You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have been using this method of initialization for a while as it performs better than default or orthogonal. I thought it would make sens to add this to the public repo.sense
🚀 Feature
Add a third option for weight_init as 'kaiming he' (see https://pytorch.org/docs/stable/nn.init.html and https://arxiv.org/pdf/1803.08494).
Motivation
I have been using this method of initialization for a while as it performs better than default or orthogonal. I thought it would make sens to add this to the public repo.sense
Pitch
Already wrote the code changes here.
5b08294
Alternatives
No response
Additional context
Tests do not currently exist for layer initialization, so I haven't added any for such a simple addition.
Checklist
The text was updated successfully, but these errors were encountered: