add deprecated for softmax_with_cross_entropy #31722

chajchaj · 2021-03-18T10:08:20Z

PR types

Function optimization

PR changes

APIs

Describe

1、softmax_with_cross_entropy是历史原因保留的api，2.0下并不推荐用户使用，所以增加打印deprecated警告
2、完善nn/functional/loss.py中cross_entropy的英文文档
3、完善nn/layer/loss.py中CrossEntropyLoss的英文文档
4、修复softlabel为true，并指定weight参数时cross_entropy报错问题
5、python端加开关softmax_switch

paddle-bot-old · 2021-03-18T10:08:22Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

…evelop

XiaoguangHu01 · 2021-03-23T06:16:45Z

python/paddle/fluid/layers/loss.py

@@ -1162,7 +1162,8 @@ def softmax_with_cross_entropy(logits,
                               ignore_index=kIgnoreIndex,
                               numeric_stable_mode=True,
                               return_softmax=False,
-                               axis=-1):
+                               axis=-1,
+                               softmax_switch=True):


softmax_with_cross_entropy是待废弃的api，不用升级，建议不修改这部分fluid下的python代码。
新功能可以在paddle.nn.functional.loss.py文件，通过调用底层core.ops.softmax_with_cross_entropy和append op的方式实现，避免对待废弃代码的依赖。

XiaoguangHu01 · 2021-03-23T06:47:22Z

python/paddle/nn/functional/loss.py

@@ -1119,94 +1137,203 @@ def cross_entropy(input,
                  reduction='mean',
                  soft_label=False,
                  axis=-1,
+                  softmax_switch=True,


switch这个单词有转换的意思，在这里可能不太直观；
建议用use_softmax=True或者enable_softmax=True，类似use_cudnn，use_global_stat

…ch to use_softmax 3) add example for softlabel for cross_entropy, test=develop

jzhang533

文档中的问题还麻烦改进。

jzhang533 · 2021-03-23T08:33:16Z

python/paddle/nn/functional/loss.py

-    Because this operator performs a softmax on logits internally, it expects
-    unscaled logits. This operator should not be used with the output of
-    softmax operator since that would produce incorrect results.
+    to provide a more numerically stable computing.


加了use_softmax这个参数后，这一段描述也相应的更新一下吧。这里只描述了default的行为。

jzhang533 · 2021-03-23T08:37:00Z

python/paddle/nn/functional/loss.py

-    The equation is as follows:
+    This operator can be used to calculate the softmax cross entropy loss with soft and hard labels.
+    Where, the hard labels mean the actual label value, 0, 1, 2, etc.  And the soft labels 
+    mean the probability of the actual label, 0.6, 0.8, 0.2, etc.


碰到过好几个用户说看不懂这里是在说什么意思。看看还能不能优化一下，至少体现出来hard label时的值的范围是[0, C-1]， soft label的时候给的李孜的和为1.0。

jzhang533 · 2021-03-23T08:37:55Z

python/paddle/nn/functional/loss.py


-        loss_j =  -\\text{logits}_{label_j} +
-        \\log\\left(\\sum_{i=0}^{K}\\exp(\\text{logits}_i)\\right), j = 1,..., K
+        1. Hard label (each sample can only be assigned into one category)


use_softmax=False时的公式，是否要体现出来。

jzhang533 · 2021-03-23T08:39:13Z

python/paddle/nn/functional/loss.py

+	    :math:`[N_1, N_2, ..., N_k, C]`, where C is number of classes ,  ``k >= 1`` . 
+            Note: it expects unscaled logits. This operator should not be used with the 
+            output of softmax operator, which will produce incorrect results.
+


use_softmax = False时的输入，可以是softmax的结果。

jzhang533 · 2021-03-23T08:39:42Z

python/paddle/nn/functional/loss.py

+            reduction='mean'
+            input_np = np.random.random([N, C]).astype(np.float64)  
+            label_np = np.random.randint(0, C, size=(N)).astype(np.int64)  
+            weight_np = np.random.random([C]).astype(np.float64)  


构造示例的时候可以直接用paddle.rand

jzhang533 · 2021-03-23T08:40:23Z

python/paddle/nn/functional/loss.py

+
+        2. reduction
+
+            2.1 if the ``reduction`` parameter is ``none`` 


none建议改为None。

已修改weight为None

jzhang533 · 2021-03-23T08:41:36Z

python/paddle/nn/layer/loss.py

+            labels = np.random.uniform(0.1, 1.0, shape).astype(dtype)
+            labels /= np.sum(labels, axis=axis, keepdims=True)
+            paddle.set_device("cpu")
+            paddle.disable_static()


为什么需要调用paddle.disable_static?

…est=develop

XiaoguangHu01

LGTM

jzhang533 · 2021-03-27T04:02:23Z

python/paddle/nn/functional/loss.py

+        .. code-block:: python
+
+            import paddle
+            import numpy as np


这行不需要吧。

jzhang533 · 2021-03-27T04:04:44Z

python/paddle/nn/functional/loss.py

+            shape = [N, C]
+            reduction='mean'
+            weight = None
+            logits = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0, seed=99999)


这里为啥没像example1一样，用paddle.seed设置种子呢？

jzhang533 · 2021-03-27T04:05:03Z

python/paddle/nn/functional/loss.py

+            logits = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0, seed=99999)
+            labels = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0, seed=99999)
+            labels /= paddle.sum(labels, axis=axis, keepdim=True)
+            paddle.set_device("cpu")


这行是不是也不需要？

jzhang533 · 2021-03-27T04:12:13Z

python/paddle/nn/functional/loss.py

+        - **soft_label** (bool, optional)
+
+            Indicate whether label is soft. 
+            If soft_label=False, the label is hard.  If soft_label=True, the label is soft.


这句话好奇怪呀。

jzhang533 · 2021-03-27T04:13:27Z

python/paddle/nn/functional/loss.py

+        - **axis** (int, optional)
+
+            The index of dimension to perform softmax calculations. 
+            It should be in range :math:`[-1, rank - 1]`, where :math:`rank` is the rank of 


rank 可改为number of dimensions 或者缩写ndim。（可参考API的规范）

…y, test=develop

jzhang533

lgtm

* add deprecated for softmax_with_cross_entropy, test=develop * test for deprecated in english doc, test=develop * test deprecated for softmax_with_cross_entropy in english doc, test=develop * fix readme and English doc for cross_entropy, test=develop * rm test for softmax_with_cross_entropy deprecated, test=develop * update readme for CrossEntropyLoss, test=develop * fix readme format, test=develop * fix readme format, test=develop * fix readme format for cross_entropy, test=develop * add softmax_switch and fix softlabel for cross_entropy, test=develop * 1)recovery softmax_with_cross_entropy in fluid 2) change softmax_switch to use_softmax 3) add example for softlabel for cross_entropy, test=develop * fix Example number for cross_entropy, test=develop * fix code format, test=develop * fix for CI-Coverage, test=develop * fix for CI-Coverage, test=develop * fix ci-coverage for Non-ASCII character '\xe2' in file, test=develop * fix ci-coverage for Non-ASCII character '\xe2' in nn.layer.loss.py, test=develop * update description for doc when use_softmax=Fasle, test=develop * fix some docs and code example for cross_entropy, test=develop * delete redundant description for soft_label parameter of cross_entropy, test=develop * fix some comment for test_cross_entropy_loss.py, test=develop

* add deprecated for softmax_with_cross_entropy (#31722) * add deprecated for softmax_with_cross_entropy, test=develop * test for deprecated in english doc, test=develop * test deprecated for softmax_with_cross_entropy in english doc, test=develop * fix readme and English doc for cross_entropy, test=develop * rm test for softmax_with_cross_entropy deprecated, test=develop * update readme for CrossEntropyLoss, test=develop * fix readme format, test=develop * fix readme format, test=develop * fix readme format for cross_entropy, test=develop * add softmax_switch and fix softlabel for cross_entropy, test=develop * 1)recovery softmax_with_cross_entropy in fluid 2) change softmax_switch to use_softmax 3) add example for softlabel for cross_entropy, test=develop * fix Example number for cross_entropy, test=develop * fix code format, test=develop * fix for CI-Coverage, test=develop * fix for CI-Coverage, test=develop * fix ci-coverage for Non-ASCII character '\xe2' in file, test=develop * fix ci-coverage for Non-ASCII character '\xe2' in nn.layer.loss.py, test=develop * update description for doc when use_softmax=Fasle, test=develop * fix some docs and code example for cross_entropy, test=develop * delete redundant description for soft_label parameter of cross_entropy, test=develop * fix some comment for test_cross_entropy_loss.py, test=develop * cherry-pick for add deprecated,test=develop

add deprecated for softmax_with_cross_entropy, test=develop

86b4f09

chajchaj added 9 commits March 18, 2021 10:33

test for deprecated in english doc, test=develop

5ae2042

test deprecated for softmax_with_cross_entropy in english doc, test=d…

48e0af0

…evelop

fix readme and English doc for cross_entropy, test=develop

8d1a611

rm test for softmax_with_cross_entropy deprecated, test=develop

799b9fe

update readme for CrossEntropyLoss, test=develop

a81d043

fix readme format, test=develop

6127f30

fix readme format, test=develop

aa50d2a

fix readme format for cross_entropy, test=develop

3778297

add softmax_switch and fix softlabel for cross_entropy, test=develop

f8a2b7b

XiaoguangHu01 reviewed Mar 23, 2021

View reviewed changes

chajchaj added 2 commits March 23, 2021 07:48

1)recovery softmax_with_cross_entropy in fluid 2) change softmax_swit…

04b778c

…ch to use_softmax 3) add example for softlabel for cross_entropy, test=develop

fix Example number for cross_entropy, test=develop

43278ad

jzhang533 reviewed Mar 23, 2021

View reviewed changes

chajchaj added 6 commits March 24, 2021 06:06

fix code format, test=develop

b3180de

fix for CI-Coverage, test=develop

8155524

fix for CI-Coverage, test=develop

99c74e1

fix ci-coverage for Non-ASCII character '\xe2' in file, test=develop

ecf0735

fix ci-coverage for Non-ASCII character '\xe2' in nn.layer.loss.py, t…

112f202

…est=develop

update description for doc when use_softmax=Fasle, test=develop

38f0b6b

XiaoguangHu01 previously approved these changes Mar 26, 2021

View reviewed changes

chalsliu previously approved these changes Mar 26, 2021

View reviewed changes

jzhang533 reviewed Mar 27, 2021

View reviewed changes

raindrops2sea previously approved these changes Mar 28, 2021

View reviewed changes

fix some docs and code example for cross_entropy, test=develop

be38f31

chajchaj dismissed stale reviews from raindrops2sea, chalsliu, and XiaoguangHu01 via be38f31 March 29, 2021 02:34

delete redundant description for soft_label parameter of cross_entrop…

5e6d703

…y, test=develop

jzhang533 previously approved these changes Mar 29, 2021

View reviewed changes

fix some comment for test_cross_entropy_loss.py, test=develop

3018e72

chajchaj dismissed jzhang533’s stale review via 3018e72 March 29, 2021 03:07

raindrops2sea approved these changes Mar 29, 2021

View reviewed changes

chalsliu approved these changes Mar 29, 2021

View reviewed changes

XiaoguangHu01 merged commit 73a6fa3 into PaddlePaddle:develop Mar 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add deprecated for softmax_with_cross_entropy #31722

add deprecated for softmax_with_cross_entropy #31722

chajchaj commented Mar 18, 2021 •

edited

Loading

paddle-bot-old bot commented Mar 18, 2021

XiaoguangHu01 Mar 23, 2021

chajchaj Mar 26, 2021

XiaoguangHu01 Mar 23, 2021

chajchaj Mar 26, 2021

jzhang533 left a comment

jzhang533 Mar 23, 2021

chajchaj Mar 26, 2021

jzhang533 Mar 23, 2021

chajchaj Mar 26, 2021

jzhang533 Mar 23, 2021

chajchaj Mar 26, 2021

jzhang533 Mar 23, 2021

chajchaj Mar 26, 2021

jzhang533 Mar 23, 2021

chajchaj Mar 26, 2021

jzhang533 Mar 23, 2021

chajchaj Mar 25, 2021

jzhang533 Mar 23, 2021

chajchaj Mar 24, 2021

XiaoguangHu01 left a comment

jzhang533 Mar 27, 2021

jzhang533 Mar 27, 2021

jzhang533 Mar 27, 2021

jzhang533 Mar 27, 2021

jzhang533 Mar 27, 2021

jzhang533 left a comment

add deprecated for softmax_with_cross_entropy #31722

add deprecated for softmax_with_cross_entropy #31722

Conversation

chajchaj commented Mar 18, 2021 • edited Loading

PR types

PR changes

Describe

paddle-bot-old bot commented Mar 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jzhang533 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jzhang533 left a comment

Choose a reason for hiding this comment

chajchaj commented Mar 18, 2021 •

edited

Loading