Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow tests #878

Open
zanieb opened this issue Jan 10, 2024 · 9 comments
Open

Slow tests #878

zanieb opened this issue Jan 10, 2024 · 9 comments
Labels
help wanted Contribution especially encouraged internal A refactor or improvement that is not user-facing

Comments

@zanieb
Copy link
Member

zanieb commented Jan 10, 2024

Last updated for commit 14d535f [source] with threshold of 10s

Windows

36.415s   uv::lock lock_conditional_dependency_extra
31.827s   uv::pip_sync require_hashes_find_links_no_hash
31.453s   uv::lock lock_sdist_url
31.110s   uv::pip_install find_links_no_binary
29.701s   uv::pip_sync require_hashes_source_no_binary
29.194s   uv::pip_install invalidate_editable_dynamic
26.146s   uv::lock lock_requires_python
25.339s   uv::pip_sync sync_editable
24.902s   uv::lock lock_dependency_extra
24.825s   uv::lock lock_project_extra
24.556s   uv::lock lock_dev
24.321s   uv::venv create_venv_unknown_python_patch
24.302s   uv::run run_with_python_version
22.766s   uv::pip_install install_git_private_https_pat_at_ref
22.327s   uv::pip_install tool_uv_sources
22.079s   uv::pip_sync install_git_subdirectories
21.839s   uv::pip_install already_installed_remote_url
20.074s   uv::lock lock_sdist_git
19.151s   uv::lock lock_wheel_registry
16.941s   uv::lock lock_wheel_url
15.760s   uv::pip_compile compile_git_mismatched_name
15.198s   uv::pip_install install_sdist_resolution_lowest
15.117s   uv::pip_sync require_hashes_at_least_one
14.605s   uv::pip_install reinstall_duplicate
14.067s   uv::lock lock_git_sha
13.984s   uv::pip_install config_settings
13.733s   uv::pip_compile generate_hashes_git
13.697s   uv::lock_scenarios fork_marker_accrue
13.654s   uv::lock_scenarios fork_basic
13.634s   uv::pip_install direct_url_zip_file_bunk_permissions
13.477s   uv::pip_install dry_run_uninstall_url_dependency
13.157s   uv::pip_compile conflicting_repeated_url_dependency_version_match
13.093s   uv::pip_sync install_build_system_no_backend
12.980s   uv::pip_sync install_numpy_py38
12.163s   uv::pip_install invalidate_editable_on_change
12.135s   uv::pip_install invalidate_path_on_change
12.000s   uv::lock_scenarios fork_marker_disjoint
11.545s   uv::pip_install only_binary_editable_setup_py
11.539s   uv::pip_install install_editable
11.483s   uv::pip_compile compile_git_concurrent_access
11.346s   uv::pip_sync find_links
11.196s   uv::pip_compile incompatible_narrowed_url_dependency
11.081s   uv::pip_sync invalidate_on_change
11.006s   uv::pip_compile unnamed_git_requirement
10.950s   uv::pip_sync install_git_source_dist_cached
10.886s   uv::pip_uninstall uninstall_by_path
10.743s   uv::pip_compile tool_uv_sources
10.738s   uv::pip_uninstall uninstall_duplicate
10.666s   uv::pip_compile compile_git_unnamed_concurrent_access
10.533s   uv::pip_compile conflicting_repeated_url_dependency
10.516s   uv::pip_compile allowed_transitive_url_dependency
10.492s   uv::pip_sync sync_legacy_sdist_pep_517
10.394s   uv::pip_install install_git_public_https
10.318s   uv::pip_compile allowed_transitive_canonical_url_dependency
10.265s   uv::pip_compile compatible_broader_url_dependency
10.264s   uv::pip_uninstall uninstall_duplicate_by_path
10.032s   uv::pip_install install_symlink

macOS

24.011s   uv::pip_install invalidate_editable_dynamic
22.594s   uv::pip_sync require_hashes_find_links_no_hash
21.249s   uv::pip_sync require_hashes_source_no_binary
18.482s   uv::pip_compile compile_git_mismatched_name
18.322s   uv::pip_install find_links_no_binary
16.656s   uv::pip_sync install_git_subdirectories
16.136s   uv::pip_install tool_uv_sources
15.033s   uv::lock lock_conditional_dependency_extra
14.905s   uv::pip_sync sync_editable
14.385s   uv::pip_install already_installed_remote_url
14.185s   uv::pip_compile generate_hashes_git
13.424s   uv::run run_with_python_version
12.609s   uv::lock lock_sdist_url
11.912s   uv::pip_compile conflicting_repeated_url_dependency_version_match
11.042s   uv::pip_sync require_hashes_at_least_one
10.924s   uv::pip_install deptry_gitignore
10.863s   uv::pip_compile compile_git_unnamed_concurrent_access
10.847s   uv::pip_install direct_url_zip_file_bunk_permissions
10.836s   uv::pip_install install_sdist_resolution_lowest
10.669s   uv::pip_install dry_run_uninstall_url_dependency
10.544s   uv::pip_install config_settings
10.091s   uv::pip_sync invalidate_on_change
10.073s   uv::pip_compile allowed_transitive_canonical_url_dependency
10.045s   uv::pip_compile compatible_broader_url_dependency

Linux

22.710s   uv::pip_install invalidate_editable_dynamic
21.342s   uv::pip_sync require_hashes_find_links_no_hash
20.539s   uv::pip_sync require_hashes_source_no_binary
16.341s   uv::pip_install find_links_no_binary
15.721s   uv::lock lock_conditional_dependency_extra
15.313s   uv::pip_sync sync_editable
13.621s   uv::run run_with_python_version
12.740s   uv::pip_sync install_git_subdirectories
12.601s   uv::pip_sync require_hashes_at_least_one
11.992s   uv::lock lock_sdist_url
11.753s   uv::pip_install already_installed_remote_url
11.027s   uv::pip_install install_sdist_resolution_lowest
11.002s   uv::pip_install dry_run_uninstall_url_dependency
10.684s   uv::pip_install tool_uv_sources
10.111s   uv::pip_install config_settings
@charliermarsh charliermarsh added the internal A refactor or improvement that is not user-facing label Jan 10, 2024
@konstin

This comment was marked as outdated.

konstin added a commit that referenced this issue Jan 12, 2024
Replace the DTLSsocket test with a dummy package that does nothing but
contain the build system specs that we need. This should speed up one of
the slowest tests.

Part of #878
konstin added a commit that referenced this issue Jan 12, 2024
Remove a test case from the `install_editable` that slows it down from
3.6s to 6.5s while providing low test coverage. It also seems to block
other tests sometimes, `cargo nextest run -E "test(editable)"
--all-features` has more consistent and lower runtimes. Surprisingly
this seems to have bigger effect than switching from pyo3 to cffi.

Used test commands:
```
rm -rf scripts/editable-installs/maturin_editable/target/ && time cargo nextest run -E "test(=install_editable)" --all-features
rm -rf scripts/editable-installs/maturin_editable/target/ && time cargo nextest run -E "test(editable)" --all-features
 ```

Part of #878
@konstin
Copy link
Member

konstin commented Jan 15, 2024

@zanieb Could you update how the original list looks for you after the recent updates?

@zanieb
Copy link
Member Author

zanieb commented Feb 13, 2024

Updated the list following #1295 — included my local results too for reference.

@charliermarsh
Copy link
Member

I think MarkupSafe contains some C code, should we replace it with something pure-Python?

@charliermarsh
Copy link
Member

I'll try real quick.

@charliermarsh
Copy link
Member

Considering moving some tests to:

@zanieb
Copy link
Member Author

zanieb commented Feb 13, 2024

Sounds good to me. Not familiar with iniconfig but anyio is great.

@zanieb
Copy link
Member Author

zanieb commented Feb 13, 2024

There's also scenario test coverage for some of these things now (like no_binary and only_binary) so we could drop the non-scenario tests if we wanted

zanieb added a commit that referenced this issue Feb 13, 2024
Show me the slow tests! ref
#878
@zanieb
Copy link
Member Author

zanieb commented Jun 10, 2024

I've updated the original post to reflect the latest changes on main. Notably this includes the use of a development drive on Windows, which should change the slow test characteristics there.

@zanieb zanieb added the help wanted Contribution especially encouraged label Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Contribution especially encouraged internal A refactor or improvement that is not user-facing
Projects
None yet
Development

No branches or pull requests

3 participants