Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[Flaky Test] Reenable two flaky test on Windows #15116

Merged
merged 1 commit into from
Jun 7, 2019

Conversation

stu1130
Copy link
Contributor

@stu1130 stu1130 commented May 31, 2019

Description

Fixes #15004
Fixes #13600
My environment: Windows 2016 Server with CUDA 10.0 + cuDNN 7.5 + MKLDNN
I could not reproduce the error using latest master(commit: 3143858)

test_l2_normalization

C:\incubator-mxnet\tests\python\unittest>nosetests --logging-level=DEBUG --verbose -s test_operator.py:test_l2_normalization 
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=683236958 to reproduce. 
[WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically *** 
test_operator.test_l2_normalization ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1163638589 to reproduce. 
ok 

---------------------------------------------------------------------- 
Ran 1 test in 2.281s 

OK 

I ran 10k times

[DEBUG] 9985 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=812676407 to reproduce. 
[DEBUG] 9986 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=2025796828 to reproduce. 
[DEBUG] 9987 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=2101832288 to reproduce. 
[DEBUG] 9988 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=445269518 to reproduce. 
[DEBUG] 9989 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=905315198 to reproduce. 
[DEBUG] 9990 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=771148112 to reproduce. 
[DEBUG] 9991 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=46635746 to reproduce. 
[DEBUG] 9992 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1900954078 to reproduce. 
[DEBUG] 9993 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1151837497 to reproduce. 
[DEBUG] 9994 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=333260159 to reproduce. 
[DEBUG] 9995 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=918969863 to reproduce. 
[DEBUG] 9996 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=795988550 to reproduce. 
[DEBUG] 9997 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=812630738 to reproduce. 
[DEBUG] 9998 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=205564357 to reproduce. 
[DEBUG] 9999 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1744386769 to reproduce. 
[DEBUG] 10000 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=950060972 to reproduce. 
ok 

---------------------------------------------------------------------- 
Ran 1 test in 22706.649s 

OK 

test_softmin

1. C:\incubator-mxnet\tests\python\unittest>nosetests --logging-level=DEBUG --verbose -s test_operator.py:test_softmin
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=114386306 to reproduce.
[WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
test_operator.test_softmin ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=350762988 to reproduce.
[17:51:49] c:\incubator-mxnet\src\operator\../common/utils.h:450: MXNET_SAFE_ACCUMULATION=1 is recommended for softmax with float16 inputs. See https://mxnet.incubator.apache.org/versions/master/faq/env_var.html for more details.
ok

----------------------------------------------------------------------
Ran 1 test in 0.234s

OK

2. C:\incubator-mxnet\tests\python\unittest>nosetests --logging-level=DEBUG --verbose -s test_operator.py:test_softmin
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=855097121 to reproduce.
[WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
test_operator.test_softmin ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=872415515 to reproduce.
[17:53:50] c:\incubator-mxnet\src\operator\../common/utils.h:450: MXNET_SAFE_ACCUMULATION=1 is recommended for softmax with float16 inputs. See https://mxnet.incubator.apache.org/versions/master/faq/env_var.html for more details.
ok

----------------------------------------------------------------------
Ran 1 test in 0.281s

OK

3. 
C:\incubator-mxnet\tests\python\unittest>nosetests --logging-level=DEBUG --verbose -s test_operator.py:test_softmin
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1589347991 to reproduce.
[WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
test_operator.test_softmin ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=644753921 to reproduce.
[17:54:53] c:\incubator-mxnet\src\operator\../common/utils.h:450: MXNET_SAFE_ACCUMULATION=1 is recommended for softmax with float16 inputs. See https://mxnet.incubator.apache.org/versions/master/faq/env_var.html for more details.
ok

----------------------------------------------------------------------
Ran 1 test in 0.172s

OK

and I ran 10k times

[DEBUG] 9984 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=599685539 to reproduce.
[DEBUG] 9985 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1665226214 to reproduce.
[DEBUG] 9986 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1114501375 to reproduce.
[DEBUG] 9987 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1971865213 to reproduce.
[DEBUG] 9988 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=950058536 to reproduce.
[DEBUG] 9989 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=924354158 to reproduce.
[DEBUG] 9990 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1184172882 to reproduce.
[DEBUG] 9991 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=101999963 to reproduce.
[DEBUG] 9992 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1243690743 to reproduce.
[DEBUG] 9993 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1696032781 to reproduce.
[DEBUG] 9994 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1869141094 to reproduce.
[DEBUG] 9995 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=353658520 to reproduce.
[DEBUG] 9996 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=353012484 to reproduce.
[DEBUG] 9997 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=765072115 to reproduce.
[DEBUG] 9998 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1216702949 to reproduce.
[DEBUG] 9999 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=678578924 to reproduce.
[DEBUG] 10000 of 10000: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=169040545 to reproduce.
ok

----------------------------------------------------------------------
Ran 1 test in 1491.462s

OK

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Ping @perdasilva @marcoabreu for review

Comments

@stu1130 stu1130 changed the title Reenable two flaky test on Windows [Flaky Test] Reenable two flaky test on Windows May 31, 2019
@abhinavs95
Copy link
Contributor

@mxnet-label-bot add [Test, Windows, pr-awaiting-review]

Copy link
Member

@wkcn wkcn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! LGTM.

Thank you!

@stu1130 stu1130 closed this Jun 3, 2019
@stu1130 stu1130 reopened this Jun 4, 2019
@sandeep-krishnamurthy sandeep-krishnamurthy merged commit bcff498 into apache:master Jun 7, 2019
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review Test Windows
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Flaky Test] test_operator.test_l2_normalization Flaky test test_softmin failed CI
5 participants