Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve GHA conda setup performance #2701

Merged
merged 18 commits into from
Jan 20, 2023
Merged

Conversation

jsiirola
Copy link
Member

@jsiirola jsiirola commented Jan 20, 2023

Fixes # .

Summary/Motivation:

This PR reworks how the GHA jobs configure conda environments, with an eye to reducing the overall GHA test time. This makes two significant changes:

  • use a consistent set of conda channels instead of specifying a single channel for each conda install command. This prevents unnecessary rework (e.g., installing a package and then later upgrading it from a different channel)
  • use mamba to solve the environment, if possible. This improved solver can be significantly more efficient than the default conda solver.

Preliminary testing shaves ~20 minutes off the win/3.9 job and more than an hour off the problematic win/3.11 job.

Changes proposed in this PR:

  • use mamba to solve the environment, if possible
  • set a global list of channels (with consistent priority)

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@jsiirola jsiirola requested a review from mrmundt January 20, 2023 05:01
@codecov
Copy link

codecov bot commented Jan 20, 2023

Codecov Report

Base: 87.06% // Head: 87.04% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (64b0e48) compared to base (1c8c5ab).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2701      +/-   ##
==========================================
- Coverage   87.06%   87.04%   -0.02%     
==========================================
  Files         757      757              
  Lines       84596    84596              
==========================================
- Hits        73652    73639      -13     
- Misses      10944    10957      +13     
Flag Coverage Δ
linux 84.45% <ø> (ø)
osx 74.50% <ø> (ø)
other 84.63% <ø> (ø)
win 81.78% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pyomo/contrib/pyros/pyros_algorithm_methods.py 93.37% <0.00%> (-1.81%) ⬇️
pyomo/solvers/plugins/solvers/xpress_direct.py 71.75% <0.00%> (-1.07%) ⬇️
pyomo/contrib/pyros/util.py 84.82% <0.00%> (-0.35%) ⬇️
pyomo/contrib/pyros/master_problem_methods.py 88.07% <0.00%> (-0.31%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mrmundt
Copy link
Contributor

mrmundt commented Jan 20, 2023

TIL about mamba. Thanks for teaching me something!

Copy link
Contributor

@mrmundt mrmundt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a grammar snark, but besides that, it's good.

.github/workflows/test_pr_and_main.yml Outdated Show resolved Hide resolved
.github/workflows/test_branches.yml Outdated Show resolved Hide resolved
@jsiirola jsiirola merged commit 517060e into Pyomo:main Jan 20, 2023
@jsiirola jsiirola deleted the conda-mamba-solver branch January 20, 2023 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants