【PIR Dist Op Reg No.22】 reg pull_sparse_v2 #63014

xiaoyewww · 2024-03-25T15:22:54Z

PR Category

Execute Infrastructure

PR Types

Devs

Description

#60436
注册算子pull_sparse_v2

paddle-bot · 2024-03-25T15:23:01Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

paddle-bot · 2024-03-25T15:23:03Z

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

paddle/phi/api/yaml/op_compat.yaml

paddle/fluid/pir/dialect/operator/ir/ops.yaml

xiaoyewww · 2024-03-28T15:29:39Z

@xingmingyyj 这个算子看上去还有问题，我看这个算子好像跟其他不太一样，yaml仿照push_sparse_v2来注册的

xingmingyyj · 2024-03-29T02:06:48Z

@xingmingyyj 这个算子看上去还有问题，我看这个算子好像跟其他不太一样，yaml仿照push_sparse_v2来注册的

哦哦，这个我也不清楚。麻烦研发老师看一下吧@kangguangli

paddle/fluid/pir/dialect/operator/ir/ops.yaml

paddle/phi/api/yaml/backward.yaml

paddle/phi/infermeta/binary.h

paddle/phi/infermeta/binary.cc

xiaoyewww · 2024-04-09T15:08:00Z

@kangguangli 麻烦辛苦再review一下，这里ci上报了第三方库相应的报错
#62935
看上去这两个pr是同一个问题

kangguangli · 2024-04-10T07:18:48Z

paddle/phi/infermeta/binary.cc

+                    phi::errors::InvalidArgument(
+                        "Output(Out) of PullSparseV2Op can not be null"));
+
+  auto hidden_size = embeddingdim;


Suggested change

auto hidden_size = embeddingdim;

auto hidden_size = embedding_dim;

这个PR的问题目前是出在这里。

kangguangli · 2024-04-10T07:19:20Z

paddle/phi/api/yaml/backward.yaml

+  forward : pull_sparse_v2 (Tensor[] ids, Tensor[] w, int embeddingdim = 11, int tableid = 0, str accessorclass = "", str ctrlabelname = "", int paddingid = 0, bool scalesparsegrad = true, str[] inputnames = {}, bool is_distributed = true) -> Tensor[](out)
+  args : (Tensor[] ids, Tensor[] w, Tensor[] out_grad, int embeddingdim, int tableid, str accessorclass, str ctrlabelname, int paddingid, bool scalesparsegrad, str[] inputnames, bool is_distributed)
+  output : Tensor[](out_grad_out)
+  invoke : push_sparse_v2(ids, w, out_grad, embeddingdim, tableid, accessorclass, ctrlabelname, paddingid, scalesparsegrad, inputnames, is_distributed)


这里的命名也需要替换下。

paddle-ci-bot · 2024-04-19T03:14:42Z

Sorry to inform you that 759b953's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

xingmingyyj · 2024-04-21T13:26:51Z

paddle/phi/api/yaml/op_compat.yaml

@@ -2653,6 +2653,14 @@
  outputs :
    out : Out

+- op : pull_sparse_v2
+  inputs :
+    { ids : Ids, W : w}


Suggested change

{ ids : Ids, W : w}

{ ids : Ids, w : W}

push_sparse_v2也需要同步修改一下。

xingmingyyj · 2024-04-21T13:27:27Z

paddle/phi/infermeta/binary.cc

+
+  for (size_t i = 0; i < n_ids; ++i) {
+    out[i]->set_dims(outs_dims[i]);
+    out[i]->share_lod(*ids[i], i);


这里需要设置一下dtype

xingmingyyj · 2024-04-21T13:36:38Z

test/ir/pir/translator/test_pull_sparse_v2_translator.py

+from paddle.base.layer_helper import LayerHelper
+
+
+class TestPullGpupsSparseOpTranslator(


Suggested change

class TestPullGpupsSparseOpTranslator(

class TestPullSparseV2OpTranslator(

xingmingyyj · 2024-04-21T13:37:16Z

test/ir/pir/translator/test_pull_sparse_v2_translator.py

+            inputs={"Ids": [ids], "W": [w]},
+            outputs={"Out": [out]},
+        )
+


Suggested change

return out

这里也需要将out返回

xingmingyyj · 2024-04-21T14:53:38Z

paddle/phi/api/yaml/backward.yaml

@@ -1835,6 +1835,12 @@
    data_type : x
  optional : boxes_num

+- backward_op : pull_sparse_v2_grad
+  forward : pull_sparse_v2 (Tensor[] ids, Tensor[] w, int embedding_dim = 11, int table_id = 0, str accessor_class = "", str ctrlabel_name = "", int padding_id = 0, bool scale_sparse_grad = true, str[] input_names = {}, bool is_distributed = true) -> Tensor[](out)
+  args : (Tensor[] ids, Tensor[] w, Tensor[] out_grad, int embedding_dim, int table_id, str accessor_class, str ctrlabel_name, int padding_id, bool scale_sparse_grad, str[] input_names, bool is_distributed)


Suggested change

args : (Tensor[] ids, Tensor[] w, Tensor[] out_grad, int embedding_dim, int table_id, str accessor_class, str ctrlabel_name, int padding_id, bool scale_sparse_grad, str[] input_names, bool is_distributed)

args : (Tensor[] ids, Tensor[] w, Tensor[] out_grad, int embedding_dim = 11, int table_id = 0, str accessor_class = "", str ctrlabel_name = "", int padding_id = 0, bool scale_sparse_grad = true, str[] input_names = {}, bool is_distributed = true)

现在的报错是这里造成的，需要设置一下默认值，否则python脚本就用None填充了。

感谢～已修改

xingmingyyj · 2024-04-23T02:02:04Z

现在的报错是因为out的stop_gradient属性为True,导致反向op没插入造成的，可以等pull_gpups_sparse这个PR合入之后再rerun一下CI。

xiaoyewww · 2024-04-25T15:10:48Z

现在的报错是因为out的stop_gradient属性为True,导致反向op没插入造成的，可以等pull_gpups_sparse这个PR合入之后再rerun一下CI。

多谢，目前已经rebase后重新rerun了

* feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2 * feat(pir): reg pull_sparse_v2

paddle-bot bot added the contributor External developers label Mar 25, 2024

luotao1 mentioned this pull request Mar 26, 2024

【PIR】PIR下的分布式算子注册 #60436

Closed

luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Mar 26, 2024

luotao1 assigned luotao1 and kangguangli Mar 26, 2024

xingmingyyj reviewed Mar 26, 2024

View reviewed changes

paddle/phi/api/yaml/op_compat.yaml Outdated Show resolved Hide resolved

xiaoyewww force-pushed the pir/pull_sparse_v2 branch from 36c57b0 to cc3882b Compare March 27, 2024 14:57

xingmingyyj reviewed Mar 28, 2024

View reviewed changes

paddle/fluid/pir/dialect/operator/ir/ops.yaml Outdated Show resolved Hide resolved

kangguangli reviewed Mar 29, 2024

View reviewed changes

paddle/fluid/pir/dialect/operator/ir/ops.yaml Outdated Show resolved Hide resolved

kangguangli reviewed Apr 7, 2024

View reviewed changes

paddle/phi/api/yaml/backward.yaml Outdated Show resolved Hide resolved

xiaoyewww force-pushed the pir/pull_sparse_v2 branch from edea9e7 to 015a785 Compare April 7, 2024 11:16

kangguangli reviewed Apr 7, 2024

View reviewed changes

paddle/phi/infermeta/binary.h Outdated Show resolved Hide resolved

paddle/phi/infermeta/binary.cc Outdated Show resolved Hide resolved

kangguangli reviewed Apr 10, 2024

View reviewed changes

xingmingyyj reviewed Apr 21, 2024

View reviewed changes

kangguangli requested review from kangguangli and xingmingyyj April 22, 2024 03:10

xiaoyewww added 7 commits April 25, 2024 15:00

feat(pir): reg pull_sparse_v2

09e7125

feat(pir): reg pull_sparse_v2

015c6de

feat(pir): reg pull_sparse_v2

b72ff80

feat(pir): reg pull_sparse_v2

024ac5e

feat(pir): reg pull_sparse_v2

535b37e

feat(pir): reg pull_sparse_v2

1f9a308

feat(pir): reg pull_sparse_v2

89fae9a

xiaoyewww added 6 commits April 25, 2024 15:01

feat(pir): reg pull_sparse_v2

30ea6f5

feat(pir): reg pull_sparse_v2

7dc0c42

feat(pir): reg pull_sparse_v2

886e6be

feat(pir): reg pull_sparse_v2

cd2153b

feat(pir): reg pull_sparse_v2

fa89685

feat(pir): reg pull_sparse_v2

1b49103

xiaoyewww force-pushed the pir/pull_sparse_v2 branch from 4641b5f to 1b49103 Compare April 25, 2024 15:10

feat(pir): reg pull_sparse_v2

109d731

xingmingyyj approved these changes Apr 26, 2024

View reviewed changes

kangguangli approved these changes Apr 26, 2024

View reviewed changes

zyfncg approved these changes Apr 26, 2024

View reviewed changes

kangguangli merged commit 2566968 into PaddlePaddle:develop Apr 26, 2024
28 of 30 checks passed

xiaoyewww deleted the pir/pull_sparse_v2 branch May 10, 2024 15:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【PIR Dist Op Reg No.22】 reg pull_sparse_v2 #63014

【PIR Dist Op Reg No.22】 reg pull_sparse_v2 #63014

xiaoyewww commented Mar 25, 2024 •

edited

Loading

paddle-bot bot commented Mar 25, 2024

paddle-bot bot commented Mar 25, 2024

xiaoyewww commented Mar 28, 2024

xingmingyyj commented Mar 29, 2024 •

edited

Loading

xiaoyewww commented Apr 9, 2024 •

edited

Loading

kangguangli Apr 10, 2024

kangguangli Apr 10, 2024

paddle-ci-bot bot commented Apr 19, 2024

xingmingyyj Apr 21, 2024 •

edited

Loading

xingmingyyj Apr 21, 2024

xingmingyyj Apr 21, 2024

xingmingyyj Apr 21, 2024

xingmingyyj Apr 21, 2024

xiaoyewww Apr 22, 2024

xingmingyyj commented Apr 23, 2024

xiaoyewww commented Apr 25, 2024

	auto hidden_size = embeddingdim;
	auto hidden_size = embedding_dim;

		from paddle.base.layer_helper import LayerHelper


		class TestPullGpupsSparseOpTranslator(

	class TestPullGpupsSparseOpTranslator(
	class TestPullSparseV2OpTranslator(

	args : (Tensor[] ids, Tensor[] w, Tensor[] out_grad, int embedding_dim, int table_id, str accessor_class, str ctrlabel_name, int padding_id, bool scale_sparse_grad, str[] input_names, bool is_distributed)
	args : (Tensor[] ids, Tensor[] w, Tensor[] out_grad, int embedding_dim = 11, int table_id = 0, str accessor_class = "", str ctrlabel_name = "", int padding_id = 0, bool scale_sparse_grad = true, str[] input_names = {}, bool is_distributed = true)

【PIR Dist Op Reg No.22】 reg pull_sparse_v2 #63014

【PIR Dist Op Reg No.22】 reg pull_sparse_v2 #63014

Conversation

xiaoyewww commented Mar 25, 2024 • edited Loading

PR Category

PR Types

Description

paddle-bot bot commented Mar 25, 2024

paddle-bot bot commented Mar 25, 2024

xiaoyewww commented Mar 28, 2024

xingmingyyj commented Mar 29, 2024 • edited Loading

xiaoyewww commented Apr 9, 2024 • edited Loading

kangguangli Apr 10, 2024

Choose a reason for hiding this comment

kangguangli Apr 10, 2024

Choose a reason for hiding this comment

paddle-ci-bot bot commented Apr 19, 2024

xingmingyyj Apr 21, 2024 • edited Loading

Choose a reason for hiding this comment

xingmingyyj Apr 21, 2024

Choose a reason for hiding this comment

xingmingyyj Apr 21, 2024

Choose a reason for hiding this comment

xingmingyyj Apr 21, 2024

Choose a reason for hiding this comment

xingmingyyj Apr 21, 2024

Choose a reason for hiding this comment

xiaoyewww Apr 22, 2024

Choose a reason for hiding this comment

xingmingyyj commented Apr 23, 2024

xiaoyewww commented Apr 25, 2024

xiaoyewww commented Mar 25, 2024 •

edited

Loading

xingmingyyj commented Mar 29, 2024 •

edited

Loading

xiaoyewww commented Apr 9, 2024 •

edited

Loading

xingmingyyj Apr 21, 2024 •

edited

Loading