Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-106300: Improve assertRaises(Exception) usage in test_runner #106737

Merged
merged 1 commit into from
Aug 16, 2023

Conversation

sobolevn
Copy link
Member

@sobolevn sobolevn commented Jul 14, 2023

Explaining changes:

  1. I've refactored all raise Exception to raise our new CustomError, so we can use it with self.assertRaises(CustomError)
  2. To fix string-based tests I've added CustomErrorRepr helper, so it would work for two styles of test execution (direct and via regtest):
(.venv) ~/Desktop/cpython  main ✗                                                         
» ./python.exe -m test test_unittest.test_runner    
0:00:00 load avg: 1.21 Run tests sequentially
0:00:00 load avg: 1.21 [1/1] test_unittest.test_runner

== Tests result: SUCCESS ==

1 test OK.

Total duration: 253 ms
Tests result: SUCCESS
                                                                                           
(.venv) ~/Desktop/cpython  main ✗                                                         
» ./python.exe Lib/test/test_unittest/test_runner.py
..................................................
----------------------------------------------------------------------
Ran 50 tests in 0.225s

OK
  1. I've fixed the bug from assertRaises(Regex)?(Exception) is problematic #106300 (comment) Now TestableTest.doClassCleanups() is not expected to raise any exceptions :)

Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On one hand, most of these changes are not necessary, because exceptions are differentiated by their messages.

On other hand, a bug similar to the bug in doClassCleanups() testing may occur again, and a custom exception will help catch it.

LGTM. Thank you.

@serhiy-storchaka serhiy-storchaka added needs backport to 3.11 only security fixes needs backport to 3.12 bug and security fixes labels Aug 16, 2023
@serhiy-storchaka serhiy-storchaka merged commit fd9d70a into python:main Aug 16, 2023
22 checks passed
@miss-islington
Copy link
Contributor

Thanks @sobolevn for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-108006 is a backport of this pull request to the 3.12 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 16, 2023
…ythonGH-106737)

Use a custom exception to prevent unintentional silence of actual errors.
(cherry picked from commit fd9d70a)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@bedevere-bot bedevere-bot removed the needs backport to 3.12 bug and security fixes label Aug 16, 2023
@bedevere-bot
Copy link

GH-108007 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Aug 16, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 16, 2023
…ythonGH-106737)

Use a custom exception to prevent unintentional silence of actual errors.
(cherry picked from commit fd9d70a)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
serhiy-storchaka pushed a commit that referenced this pull request Aug 16, 2023
…GH-106737) (GH-108007)

Use a custom exception to prevent unintentional silence of actual errors.
(cherry picked from commit fd9d70a)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Yhg1s pushed a commit that referenced this pull request Aug 16, 2023
…GH-106737) (#108006)

gh-106300: Improve errors testing in test_unittest.test_runner (GH-106737)

Use a custom exception to prevent unintentional silence of actual errors.
(cherry picked from commit fd9d70a)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants