-
Notifications
You must be signed in to change notification settings - Fork 522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Output python-versions
, containing all Python versions, for cache-busting
#611
Conversation
pythonVersions.push( | ||
`${installed.resolvedPythonVersion}-pypy${installed.resolvedPyPyVersion}` | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to change pypy and python version parts order ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with either order.
I did it this way for potential parsing purposes later in the workflow. It seemed like if someone wanted to determine all of the Python versions (regardless of CPython vs PyPy implementation), it would be easier if the Python version appeared at the beginning of the string. If -pypy
appeared later in the string it could be stripped off or extracted out as needed.
However, I don't view the ordering as a critical need, and can change the order if desired.
core.info(`Successfully set up ${installed.impl} (${pythonVersion})`); | ||
} | ||
} | ||
core.setOutput('python-versions', pythonVersions.sort().join(',')); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to sort versions ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I sorted it to help stabilize the cache key. If one developer writes their workflow YAML file as:
- uses: actions/setup-python@v4
with:
python-version: |
3.11
3.10
3.9
3.8
3.7
and the versions are later rearranged to:
- uses: actions/setup-python@v4
with:
python-version: |
3.7
3.8
3.9
3.10
3.11
sorting the versions will provide stability to the cache key.
I currently think this is beneficial behavior, but can remove the sorting if desired.
@dmitry-shibanov I just saw the new support for "allow-prereleases", which is fantastic! I would love to get additional feedback about this PR, or to have it merged, so that I can use this new output as a cache key. I don't know if I communicated the use case well, so to summarize: Setting up a GitHub runner for each Python version is comparatively expensive for some of my projects -- it can take 30 seconds to spin up a runner, 5 seconds to set up the tox environment, but the test suite only takes 2 seconds to run for each Python version. Therefore I install all Python versions on a single runner:
The cache needs to be invalidated if any Python version changes, and this PR makes that possible. Python 3.12 releases are coming out with frequency, and I would like to take advantage of the new "allow-prereleases" feature and have this Thanks for your time, and your work on setup-python! |
python-versions
, containing all Python versionspython-versions
, containing all Python versions, for cache-busting
@dmitry-shibanov, please let me know if you have additional feedback. I'm interested in having this PR merge for effective cache-busting when multiple Python versions are installed. |
197e69d
to
949d439
Compare
Rebased on |
949d439
to
2d235cd
Compare
I'm closing this PR because I've written a GitHub action to address my need for strong cache-busting based on installed Python versions. |
Description:
This introduces a
python-versions
output, which is a comma-separated string of all installed Python versions and implementations. This is useful for cache-busting when multiple Python versions are installed, as described in #606.For example, it is now possible to cache a
.tox/
test directory and invalidate the cache when any of the Python versions change. Caching the.tox/
directory offers a significant speedup to test setup times, but if a non-default Python's version changes (say, if 3.11.1 is the default but 3.10.x increments to 3.10.y), it is currently very difficult to invalidate the cache. This new output variable allows for trivial cache invalidation in this situation.To demonstrate the new output variable, I created a workflow with a single
setup-python
step that installs multiple CPython and PyPy versions, then usedecho
to show the value ofpython-versions
.The value displayed is:
which can be used directly in a cache key, or written to a file and included in a call to
hashFiles()
. The versions are sorted to avoid sensitivity to the order of Python versions specific in the workflow, but are sorted merely as strings for this purpose, not as semantic versions. Therefore a user can create a workflow that installs Python versions in any order they want, but thepython-versions
output string will be stable.NOTE: This work depends on #610 to remove trailing newlines in PyPy versions.
Related issue:
Closes #606
Check list: