Add forward support for PReLU #14940
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
3 times, most recently
from
dc0e900 to
5ba2908
Compare
umadevimcw
requested changes
Nov 12, 2024
For PReLU with scalar weights, sweep tests are already in place(Tensor- scalar version). The sweep tests will be updated to cover the tensor API once PReLU is implemented with support for 1D and 0D tensors. |
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
c755f5e to
fbcac9c
Compare
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
fbcac9c to
5a3afc2
Compare
mouliraj-mcw
requested review from
KalaivaniMCW,
mcw-anasuya,
Aswinmcw and
VirdhatchaniKN
umadevimcw
approved these changes
Nov 13, 2024
VirdhatchaniKN
approved these changes
Nov 13, 2024
mcw-anasuya
approved these changes
Nov 13, 2024
ttnn/cpp/ttnn/operations/eltwise/binary/device/binary_composite_op.cpp
Outdated
Show resolved
Hide resolved
ttnn/cpp/ttnn/operations/eltwise/binary/device/binary_composite_op.cpp
Outdated
Show resolved
Hide resolved
Aswinmcw
force-pushed
the
mouliraj/prelu
branch
2 times, most recently
from
4dd3df2 to
8d1bc93
Compare
mouliraj-mcw
requested review from
eyonland,
patrickroberts,
yan-zaretskiy,
ayerofieiev-tt and
razorback3
as code owners
mouliraj-mcw
requested review from
rtawfik01,
rdjogoTT and
ttmtrajkovic
as code owners
rdjogoTT
reviewed
Nov 14, 2024
tt_metal/hw/ckernels/wormhole_b0/metal/llk_api/llk_sfpu/ckernel_sfpu_prelu.h
Outdated
Show resolved
Hide resolved
|
Can you add this for Blackhole as well? |
KalaivaniMCW
approved these changes
Nov 15, 2024
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
8d1bc93 to
34c469d
Compare
rdjogoTT
reviewed
Nov 15, 2024
tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_sfpu/ckernel_sfpu_prelu.h
Outdated
Show resolved
Hide resolved
rdjogoTT
reviewed
Nov 15, 2024
tt_metal/hw/ckernels/blackhole/metal/llk_api/llk_sfpu/ckernel_sfpu_prelu.h
Show resolved
Hide resolved
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
34c469d to
905d2ee
Compare
mcw-anasuya
reviewed
Nov 15, 2024
tests/ttnn/unit_tests/operations/eltwise/test_binary_composite.py
Outdated
Show resolved
Hide resolved
rdjogoTT
reviewed
Nov 15, 2024
mcw-anasuya
approved these changes
Nov 15, 2024
rdjogoTT
approved these changes
Nov 15, 2024
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
905d2ee to
442bed9
Compare
eyonland
approved these changes
Nov 18, 2024
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
442bed9 to
54c522d
Compare
ayerofieiev-tt
approved these changes
Nov 20, 2024
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
2 times, most recently
from
0435daa to
752eba8
Compare
mouliraj-mcw
force-pushed
the
mouliraj/prelu
branch
from
752eba8 to
5103293
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ticket
Link to Github Issue #8544
Problem description
Currently PReLU is aliased to LeakyRelu which is not correct. It needs to be properly implemented in the eltwise operation.
What's changed
Additional Information
In Torch PReLU, the second input tensor can only have two valid shapes: either a tensor with a single value (1) or a tensor with a size equal to the number of input channels (default is 1). Currently, This implementation only supports cases where it matches the number of channels. Support for a single-value tensor requires additional handling at the low-level kernel.
Checklist