Add support for broadcast multiply along batch dimension (W)

[kpaigwar]

Requirement

Need support for an op which can perform element-wise multiply and broadcast along batch dim.
For example, (32, 1, 32, 1024)*(1, 1, 32, 1024)

[kpaigwar]

fyi @apalaguha @esmalTT

[kpaigwar]

This op is crtical for us when doing element-wise multiplication of conv1D weights (batch independent) with inputs (batch dependent). The workaround to manually broadcast the conv1D weights makes the implementation harder and non-performant.

[bharane-abb]

Hi @kpaigwar @jliangTT
We have come up with two ideas for multiplication without using broadcasting:

1. Using the repeat function, we will repeat the smaller tensor into the required shape and then proceed with multiplication.
2. Using the unpad function, we can unpad the larger tensor into a smaller tensor and multiply it with the smaller tensor. However, this process is time-consuming as it involves many unpad, multiply, and concat operations.

[kpaigwar]

@bharane-ab first approach would be ideal

[umadevimcw]

@kpaigwar Regarding the above why are we just considering mul operation alone? What about other binary, operations like squared_diff, add, sub, etc.. ? If we make changes in commonplace then it would be applied to all.

Tried to update the eltwise_binary_op.hpp to apply changes to all ops in binary. But getting error shown in below image. Suggested flag (in the image not helping). Not able to use repeat in the mentioned file

@kpaigwar Do you want me to go with separate op like batch_mul or some thing? Any suggestions?

[umadevimcw]

Tried the implementation in composite as a separate batch mul op #6442

[tt-aho]

If you are targeting performance, are you planning to run this model/op sharded? Or exactly as specified in this issue? The optimizations/support for sharding are different than interleaved so want to make sure we're targeting/optimizing the right thing. Similar question for #6361

[kpaigwar]

Since this issue is also related to repeat #6361, I will drop the priority of this issue and would address P0 issue first #6361

[umadevimcw]

@kpaigwar Please find the updated PR #6587 for batch mul support. It doesn't involve new op. Also changed the code in common place hence it is working for other binary op as well

[umadevimcw]

Support Merged to Main