Skip to content

Commit

Permalink
bracket
Browse files Browse the repository at this point in the history
  • Loading branch information
mcabbott authored Oct 18, 2021
1 parent eed5b64 commit b6a8964
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/activations.jl
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ const logsigmoid = logσ
Segment-wise linear approximation of tanh. Cheaper and more computational efficient version of tanh.
See [Large Scale Machine Learning](https://ronan.collobert.com/pub/matos/2004_phdthesis_lip6.pdf).
"""
hardtanh(x) = clamp(x, oftype(x, -1), oftype(x, 1) # clamp(x, -1, 1) is type-stable, but would promote Int32, for which we have tests
hardtanh(x) = clamp(x, oftype(x, -1), oftype(x, 1)) # clamp(x, -1, 1) is type-stable, but would promote Int32, for which we have tests

"""
relu(x) = max(0, x)
Expand Down

0 comments on commit b6a8964

Please sign in to comment.