Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: reduce-testing-time #3427

Merged
merged 24 commits into from
Sep 26, 2024
Merged

perf: reduce-testing-time #3427

merged 24 commits into from
Sep 26, 2024

Conversation

germa89
Copy link
Collaborator

@germa89 germa89 commented Sep 23, 2024

Description

As the title. Reducing testing time.

Issue linked

Close #3281

Checklist

@germa89 germa89 requested a review from a team as a code owner September 23, 2024 09:21
@germa89 germa89 requested review from clatapie and pyansys-ci-bot and removed request for a team September 23, 2024 09:21
@ansys-reviewer-bot
Copy link
Contributor

Thanks for opening a Pull Request. If you want to perform a review write a comment saying:

@ansys-reviewer-bot review

@germa89 germa89 self-assigned this Sep 23, 2024
@github-actions github-actions bot added the enhancement Improve any current implemented feature label Sep 23, 2024
Copy link

codecov bot commented Sep 23, 2024

Codecov Report

Attention: Patch coverage is 70.58824% with 15 lines in your changes missing coverage. Please review.

Project coverage is 85.07%. Comparing base (8298793) to head (5e037e6).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3427      +/-   ##
==========================================
- Coverage   85.14%   85.07%   -0.07%     
==========================================
  Files          55       55              
  Lines        9962     9977      +15     
==========================================
+ Hits         8482     8488       +6     
- Misses       1480     1489       +9     

@github-actions github-actions bot added CI/CD Related with CICD, Github Actions, etc maintenance General maintenance of the repo (libraries, cicd, etc) labels Sep 23, 2024
@germa89
Copy link
Collaborator Author

germa89 commented Sep 25, 2024

Ready to merge... output

Previous

Use base line: #3428

image image

local -v23.2 ubuntu

============================= slowest 10 durations =============================
82.64s call     tests/test_pool.py::TestMapdlPool::test_directory_names_function
Creating Pool:  67%|██████▋   | 2/3 [02:34<01:17, 77.13s/it]

46.45s call     tests/test_launcher.py::test_license_type_keyword
35.95s call     tests/test_launcher.py::test_license_type_additional_switch
33.84s call     tests/test_launcher.py::test_license_type_keyword_names
22.03s call     tests/test_mapdl.py::test_save_on_exit
21.41s call     tests/test_pool.py::TestMapdlPool::test_directory_names_function
20.04s call     tests/test_pool.py::TestMapdlPool::test_map_timeout
9.62s call     tests/test_pool.py::TestMapdlPool::test_heal
9.36s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart
8.77s call     tests/test_launcher.py::test_remove_temp_files_fail

Local-min: latest-ubuntu

============================= slowest 10 durations =============================
104.88s call     tests/test_launcher.py::test_license_type_additional_switch
102.53s call     tests/test_launcher.py::test_license_type_keyword_names
101.30s call     tests/test_launcher.py::test_license_type_keyword
80.61s call     tests/test_pool.py::TestMapdlPool::test_directory_names_function
23.37s call     tests/test_mapdl.py::test_save_on_exit
22.26s call     tests/test_pool.py::TestMapdlPool::test_directory_names_function
20.42s call     tests/test_launcher.py::test_remove_temp_files
19.06s call     tests/test_launcher.py::test_remove_temp_files_fail
16.76s call     tests/test_pool.py::TestMapdlPool::test_directory_names_custom_string
14.72s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart

Local: latest-ubuntu

============================= slowest 10 durations =============================
122.94s call     tests/test_launcher.py::test_license_type_keyword
101.30s call     tests/test_launcher.py::test_license_type_keyword_names
101.04s call     tests/test_launcher.py::test_license_type_additional_switch
42.14s call     tests/test_pool.py::TestMapdlPool::test_directory_names_function
26.39s call     tests/test_mapdl.py::test_save_on_exit
22.44s teardown tests/test_pool.py::TestMapdlPool::test_ip_port_n_instance[4-ip31-None-None-None-None-context31]
22.01s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart
21.96s call     tests/test_pool.py::TestMapdlPool::test_heal
21.29s call     tests/test_launcher.py::test_remove_temp_files
21.25s call     tests/test_launcher.py::test_remove_temp_files_fail
========================= PyMAPDL Pytest short summary =========================

Now

image image

Local: v23.2-ubuntu

============================= slowest 10 durations =============================
84.90s call     tests/test_pool.py::TestMapdlPool::test_only_one_instance
18.87s call     tests/test_pool.py::TestMapdlPool::test_map_timeout
10.07s call     tests/test_pool.py::TestMapdlPool::test_heal
8.86s call     tests/test_launcher.py::test_remove_temp_files_fail
8.81s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart
8.23s call     tests/test_launcher.py::test_remove_temp_files
5.96s call     tests/test_mapdl.py::test_remove_temp_dir_on_exit
5.59s call     tests/test_pool.py::TestMapdlPool::test_directory_names_function
4.44s call     tests/test_cli.py::test_launch_mapdl_cli_config
4.11s setup    tests/test_cli.py::test_launch_mapdl_cli[None]

Local-min: latest-ubuntu

============================= slowest 10 durations =============================
22.51s teardown tests/test_pool.py::TestMapdlPool::test_ip_port_n_instance[4-ip31-None-None-None-None-context31]
20.42s call     tests/test_launcher.py::test_remove_temp_files_fail
15.51s call     tests/test_launcher.py::test_remove_temp_files
14.76s call     tests/test_pool.py::TestMapdlPool::test_map_timeout
14.53s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart
14.38s call     tests/test_pool.py::TestMapdlPool::test_heal
12.61s teardown tests/test_xpl.py::test_extract
11.84s call     tests/test_pool.py::TestMapdlPool::test_only_one_instance
7.38s setup    tests/test_pool.py::TestMapdlPool::test_simple
5.28s call     tests/test_mapdl.py::test_remove_temp_dir_on_exit

Local: latest-ubuntu

============================= slowest 10 durations =============================
83.14s call     tests/test_pool.py::TestMapdlPool::test_only_one_instance
83.13s call     tests/test_pool.py::TestMapdlPool::test_only_one_instance
47.47s call     tests/test_pool.py::TestMapdlPool::test_only_one_instance
21.29s call     tests/test_launcher.py::test_remove_temp_files_fail
20.50s call     tests/test_pool.py::TestMapdlPool::test_heal
19.73s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart
18.58s call     tests/test_launcher.py::test_remove_temp_files
14.79s call     tests/test_pool.py::TestMapdlPool::test_map_timeout
8.25s call     tests/test_mapdl.py::test_remove_temp_dir_on_exit
7.01s setup    tests/test_pool.py::TestMapdlPool::test_simple

Notes

These changes reduce variability on test times.

@germa89
Copy link
Collaborator Author

germa89 commented Sep 25, 2024

I'm going to reduce also test_only_one_instance by just checking passing arguments.

@germa89
Copy link
Collaborator Author

germa89 commented Sep 25, 2024

Once CICD passes, I'm happy with the changes.

@pyansys-ci-bot LGTM.

@germa89 germa89 enabled auto-merge (squash) September 25, 2024 12:29
Copy link
Contributor

@pyansys-ci-bot pyansys-ci-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approving this PR because germa89 said so in here 😬

LGTM

@germa89
Copy link
Collaborator Author

germa89 commented Sep 26, 2024

After 5e037e6 the new times are

Local: latest-ubuntu

============================= slowest 10 durations =============================
22.54s teardown tests/test_pool.py::TestMapdlPool::test_ip_port_n_instance[4-ip31-None-None-None-None-context31]
22.35s call     tests/test_launcher.py::test_remove_temp_files_fail
16.38s call     tests/test_launcher.py::test_remove_temp_files
14.84s call     tests/test_pool.py::TestMapdlPool::test_map_timeout
14.23s call     tests/test_pool.py::TestMapdlPool::test_directory_names_default_with_restart
13.82s call     tests/test_pool.py::TestMapdlPool::test_heal
12.39s teardown tests/test_xpl.py::test_extract
5.36s call     tests/test_mapdl.py::test_remove_temp_dir_on_exit
4.65s call     tests/test_examples.py::test_download_manifold_example_data
3.92s setup    tests/test_cli.py::test_launch_mapdl_cli[None]

I could keep going... but I'm happy for the moment.

@germa89 germa89 merged commit 1a1cfd4 into main Sep 26, 2024
61 of 64 checks passed
@germa89 germa89 deleted the perf/reduce-testing-time branch September 26, 2024 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD Related with CICD, Github Actions, etc enhancement Improve any current implemented feature maintenance General maintenance of the repo (libraries, cicd, etc)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reducing testing time
2 participants