Skip to content

Commit

Permalink
#13685: Update backward embedding document (#14666)
Browse files Browse the repository at this point in the history
### Ticket
#13685 

### Problem description
- ttnn.embedding_bw doesn't support output and input gradient tensors
having different dtypes.
- ttnn.embedding_bw is not an eltwise op. So, moving corresponding tests
outside eltwise folder.

### What's changed
- Updated document to add a note mentioning input and output gradients
must have the same datatype.
- Moved sweep test from
`tests/sweep_framework/sweeps/eltwise/binary_backward/embedding_bw/embedding_bw.py`
to `tests/sweep_framework/sweeps/embedding_bw/embedding_bw.py`
- Move unit test from from
`tests/ttnn/unit_tests/operations/eltwise/backward/test_backward_embedding.py`
to `tests/ttnn/unit_tests/operations/test_backward_embedding.py`


### Updated doc
<img width="1139" alt="Screenshot 2024-11-04 at 17 49 31"
src="https://github.com/user-attachments/assets/467fec4f-cb30-4ec9-ac26-9e59827ed45b">


### Checklist
- [x] [Post commit CI
passes](https://github.com/tenstorrent/tt-metal/actions/runs/11680989159)
- PASSED
- [x] [Run
Sweeps](https://github.com/tenstorrent/tt-metal/actions/runs/11678538301)
- PASSED
- [x] [Night fast dispatch
tests](https://github.com/tenstorrent/tt-metal/actions/runs/11678522765)
(same as main)
  • Loading branch information
mcw-anasuya authored Nov 11, 2024
1 parent fbc8a9d commit fec3ebc
Show file tree
Hide file tree
Showing 4 changed files with 4 additions and 9 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/ttnn-run-sweeps.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,6 @@ on:
- eltwise.binary_backward.ldexp_bw
- eltwise.binary_backward.logaddexp_bw
- eltwise.binary_backward.logaddexp2_bw
- eltwise.binary_backward.embedding_bw.embedding_bw
- eltwise.binary_backward.addalpha_bw.addalpha_bw
- eltwise.binary_backward.subalpha_bw.subalpha_bw
- eltwise.binary_backward.xlogy_bw.xlogy_bw
Expand Down Expand Up @@ -266,6 +265,7 @@ on:
- eltwise.ternary_backward.addcmul_bw
- eltwise.ternary_backward.addcdiv_bw
- embedding.embedding
- embedding_bw.embedding_bw
- reduction.backward.prod_bw.prod_bw
- reduction.topk.topk
- reduction.argmax.argmax
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,13 @@
from functools import partial

import torch
import random
import ttnn
from tests.sweep_framework.sweep_utils.utils import gen_shapes
from tests.tt_eager.python_api_testing.sweep_tests.generation_funcs import gen_func_with_cast_tt

from tests.ttnn.utils_for_testing import check_with_pcc, start_measuring_time, stop_measuring_time
from models.utility_functions import torch_random

# Override the default timeout in seconds for hang detection.
TIMEOUT = 30

random.seed(0)


# Parameters provided to the test vector generator are defined here.
# They are defined as dict-type suites that contain the arguments to the run function as keys, and lists of possible inputs as values.
Expand Down Expand Up @@ -91,8 +85,7 @@ def run(
*,
device,
) -> list:
data_seed = random.randint(0, 20000000)
torch.manual_seed(data_seed)
torch.manual_seed(0)

batch_size, seq_length, embeddings_dim, num_embeddings = embedding_args

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ void py_bind_embedding_backward(py::module& module) {
Returns:
ttnn.Tensor: the output tensor.
Note:
The input and the output gradient tensors must have the same datatype.
Example:
>>> device_id = 0
Expand Down

0 comments on commit fec3ebc

Please sign in to comment.