Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-103224: Use the realpath of the Python executable in test_venv #103243

Merged
merged 4 commits into from
Jun 1, 2023

Conversation

artemmukhin
Copy link
Contributor

@artemmukhin artemmukhin commented Apr 4, 2023

Now running tests through a Python symlink no longer causes test_upgrade_dependencies and test_zippath_from_non_installed_posix to fail.

…env`

Now running tests through a Python symlink no longer causes `test_upgrade_dependencies` and `test_zippath_from_non_installed_posix` to fail.
@artemmukhin
Copy link
Contributor Author

@vsajip Could you please take a look? This PR only involves a small change in the test

@vsajip vsajip added skip news 🔨 test-with-buildbots Test PR w/ buildbots; report in status section labels Apr 12, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vsajip for commit 67576f5 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 12, 2023
@vsajip
Copy link
Member

vsajip commented Apr 12, 2023

In some buildbots test_venv is failing, but I haven't had time to look into why.

@arhadthedev
Copy link
Member

== Tests result: FAILURE ==
411 tests OK.
1 test failed:
    test_venv
18 tests skipped:
    test_asdl_parser test_clinic test_devpoll test_gdb test_ioctl
    test_kqueue test_launcher test_msilib test_peg_generator
    test_startfile test_tix test_tkinter test_ttk test_winconsoleio
    test_winreg test_winsound test_wmi test_zipfile64
0:06:57 load avg: 0.91
0:06:57 load avg: 0.91 Re-running failed tests in verbose mode
0:06:57 load avg: 0.91 Re-running test_venv in verbose mode (matching: test_zippath_from_non_installed_posix, test_upgrade_dependencies)
test_upgrade_dependencies (test.test_venv.BasicTest.test_upgrade_dependencies) ... FAIL
test_zippath_from_non_installed_posix (test.test_venv.BasicTest.test_zippath_from_non_installed_posix)
Test that when create venv from non-installed python, the zip path ... ERROR
======================================================================
ERROR: test_zippath_from_non_installed_posix (test.test_venv.BasicTest.test_zippath_from_non_installed_posix)
Test that when create venv from non-installed python, the zip path
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_venv.py", line 606, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd,
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/subprocess.py", line 408, in check_call
    retcode = call(*popenargs, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/subprocess.py", line 389, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/subprocess.py", line 1917, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp4fiiqaq7/bin/python3'
======================================================================
FAIL: test_upgrade_dependencies (test.test_venv.BasicTest.test_upgrade_dependencies)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_venv.py", line 237, in test_upgrade_dependencies
    builder.upgrade_dependencies(fake_context)
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/venv/__init__.py", line 459, in upgrade_dependencies
    self._call_new_python(context, '-m', 'pip', 'install', '--upgrade',
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/venv/__init__.py", line 355, in _call_new_python
    subprocess.check_output(args, **kwargs)
  File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.12/test/test_venv.py", line 222, in pip_cmd_checker
    self.assertEqual(
AssertionError: Lists differ: ['/tm[20 chars]thon3', '-m', 'pip', 'install', '--upgrade', '[14 chars]ols'] != ['/tm[20 chars]thon3.12', '-m', 'pip', 'install', '--upgrade'[17 chars]ols']
First differing element 0:
'/tmp/tmpbr5t_vk3/bin/python3'
'/tmp/tmpbr5t_vk3/bin/python3.12'
- ['/tmp/tmpbr5t_vk3/bin/python3',
+ ['/tmp/tmpbr5t_vk3/bin/python3.12',
?                               +++
   '-m',
   'pip',
   'install',
   '--upgrade',
   'pip',
   'setuptools']
----------------------------------------------------------------------
Ran 2 tests in 1.068s
FAILED (failures=1, errors=1)
test test_venv failed
1 test failed again:
    test_venv
== Tests result: FAILURE then FAILURE ==
411 tests OK.
1 test failed:
    test_venv
18 tests skipped:
    test_asdl_parser test_clinic test_devpoll test_gdb test_ioctl
    test_kqueue test_launcher test_msilib test_peg_generator
    test_startfile test_tix test_tkinter test_ttk test_winconsoleio
    test_winreg test_winsound test_wmi test_zipfile64
1 re-run test:
    test_venv
Total duration: 6 min 59 sec
Tests result: FAILURE then FAILURE
program finished with exit code 2
elapsedTime=419.107087

@artemmukhin
Copy link
Contributor Author

@arhadthedev Thank you for clarifying! I tried to fix that in the second commit. Could you please rerun the tests?

@AlexWaygood AlexWaygood added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 12, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @AlexWaygood for commit b75f881 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 12, 2023
@arhadthedev arhadthedev added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 25, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @arhadthedev for commit 205ae85 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 25, 2023
@arhadthedev
Copy link
Member

Retriggered the build because the last logs seem to be expired:

image

@vsajip vsajip merged commit 85b0b0c into python:main Jun 1, 2023
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Aug 30, 2023
… `test_venv` (pythonGH-103243)"

This reverts commit 85b0b0c.

It broke builtbots.
serhiy-storchaka added a commit that referenced this pull request Aug 30, 2023
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 Windows 3.x has failed when building commit 210a5d7.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/729/builds/5358) and take a look at the build logs.
  4. Check if the failure is related to this commit (210a5d7) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/729/builds/5358

Failed tests:

  • test.test_concurrent_futures.test_shutdown

Summary of the results of the build (if available):

== Tests result: FAILURE then SUCCESS ==

424 tests OK.

10 slowest tests:

  • test_math: 7 min 57 sec
  • test_peg_generator: 4 min 28 sec
  • test_unparse: 2 min 52 sec
  • test_tokenize: 2 min 42 sec
  • test_tarfile: 2 min 13 sec
  • test.test_multiprocessing_spawn.test_processes: 2 min 12 sec
  • test_compileall: 2 min 5 sec
  • test_unicodedata: 1 min 58 sec
  • test_fstring: 1 min 42 sec
  • test_capi: 1 min 38 sec

39 tests skipped:
test.test_asyncio.test_unix_events
test.test_multiprocessing_fork.test_manager
test.test_multiprocessing_fork.test_misc
test.test_multiprocessing_fork.test_processes
test.test_multiprocessing_fork.test_threads
test.test_multiprocessing_forkserver.test_manager
test.test_multiprocessing_forkserver.test_misc
test.test_multiprocessing_forkserver.test_processes
test.test_multiprocessing_forkserver.test_threads test_curses
test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl
test_fork1 test_gdb test_grp test_ioctl test_kqueue test_openpty
test_perf_profiler test_perfmaps test_poll test_posix test_pty
test_pwd test_readline test_resource test_syslog
test_threadsignals test_tkinter test_ttk test_wait3 test_wait4
test_xxlimited test_xxtestfuzz test_zipfile64 test_zoneinfo

1 re-run test:
test.test_concurrent_futures.test_shutdown

Total duration: 27 min 11 sec

Click to see traceback logs
Traceback (most recent call last):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\__init__.py", line 203, in _force_run
    return func(*args)
           ^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Workspace\\buildarea\\3.x.linaro-win-arm64\\build\\build\\test_python_10736�\\test_python_worker_7640�'


Traceback (most recent call last):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\__main__.py", line 2, in <module>
    main()
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\libregrtest\main.py", line 826, in main
    Regrtest().main(tests=tests, **kwargs)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\libregrtest\main.py", line 756, in main
    with os_helper.temp_cwd(test_cwd, quiet=True):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\contextlib.py", line 159, in __exit__
    self.gen.throw(value)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 531, in temp_cwd
    with temp_dir(path=name, quiet=quiet) as temp_path:
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\contextlib.py", line 159, in __exit__
    self.gen.throw(value)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 485, in temp_dir
    rmtree(path)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 442, in rmtree
    _rmtree(path)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 385, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 330, in _waitfor
    func(pathname)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 382, in _rmtree_inner
    _force_run(fullname, os.rmdir, fullname)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\__init__.py", line 214, in _force_run
    return func(*args)
           ^^^^^^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Workspace\\buildarea\\3.x.linaro-win-arm64\\build\\build\\test_python_10736�\\test_python_worker_7640�'


Traceback (most recent call last):
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 480, in temp_dir
    yield path
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\support\os_helper.py", line 533, in temp_cwd
    yield cwd_dir
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\libregrtest\main.py", line 762, in main
    self._main(tests, kwargs)
  File "C:\Workspace\buildarea\3.x.linaro-win-arm64\build\Lib\test\libregrtest\main.py", line 821, in _main
    sys.exit(0)
SystemExit: 0

carljm added a commit to carljm/cpython that referenced this pull request Aug 30, 2023
* main:
  pythongh-108520: Fix bad fork detection in nested multiprocessing use case (python#108568)
  pythongh-108590: Revert pythongh-108657 (commit 400a1ce) (python#108686)
  pythongh-108494: Argument Clinic: Document how to generate code that uses the limited C API (python#108584)
  Document Python build requirements (python#108646)
  pythongh-101100: Fix Sphinx warnings in the Logging Cookbook (python#108678)
  Fix typo in multiprocessing docs (python#108666)
  pythongh-108669: unittest: Fix documentation for TestResult.collectedDurations (python#108670)
  pythongh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (python#108657)
  Revert "pythongh-103224: Use the realpath of the Python executable in `test_venv` (pythonGH-103243)" (pythonGH-108667)
  pythongh-106320: Remove private _Py_ForgetReference() (python#108664)
  Mention Ellipsis pickling in the docs (python#103660)
  Revert "Use non alternate name for Kyiv (pythonGH-108533)" (pythonGH-108649)
  pythongh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (python#108632)
  pythongh-108455: peg_generator: install two stubs packages before running mypy (python#108637)
  pythongh-107801: Improve the accuracy of io.IOBase.seek docs (python#108268)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants