Output python-versions, containing all Python versions, for cache-busting
#611
Conversation
| pythonVersions.push( | ||
| `${installed.resolvedPythonVersion}-pypy${installed.resolvedPyPyVersion}` | ||
| ); |
There was a problem hiding this comment.
Do we need to change pypy and python version parts order ?
There was a problem hiding this comment.
I'm fine with either order.
I did it this way for potential parsing purposes later in the workflow. It seemed like if someone wanted to determine all of the Python versions (regardless of CPython vs PyPy implementation), it would be easier if the Python version appeared at the beginning of the string. If -pypy appeared later in the string it could be stripped off or extracted out as needed.
However, I don't view the ordering as a critical need, and can change the order if desired.
| core.info(`Successfully set up ${installed.impl} (${pythonVersion})`); | ||
| } | ||
| } | ||
| core.setOutput('python-versions', pythonVersions.sort().join(',')); |
There was a problem hiding this comment.
Do we need to sort versions ?
There was a problem hiding this comment.
I sorted it to help stabilize the cache key. If one developer writes their workflow YAML file as:
- uses: actions/setup-python@v4
with:
python-version: |
3.11
3.10
3.9
3.8
3.7and the versions are later rearranged to:
- uses: actions/setup-python@v4
with:
python-version: |
3.7
3.8
3.9
3.10
3.11sorting the versions will provide stability to the cache key.
I currently think this is beneficial behavior, but can remove the sorting if desired.
|
@dmitry-shibanov I just saw the new support for "allow-prereleases", which is fantastic! I would love to get additional feedback about this PR, or to have it merged, so that I can use this new output as a cache key. I don't know if I communicated the use case well, so to summarize: Setting up a GitHub runner for each Python version is comparatively expensive for some of my projects -- it can take 30 seconds to spin up a runner, 5 seconds to set up the tox environment, but the test suite only takes 2 seconds to run for each Python version. Therefore I install all Python versions on a single runner:
The cache needs to be invalidated if any Python version changes, and this PR makes that possible. Python 3.12 releases are coming out with frequency, and I would like to take advantage of the new "allow-prereleases" feature and have this Thanks for your time, and your work on setup-python! |
kurtmckee
changed the title
python-versions, containing all Python versionspython-versions, containing all Python versions, for cache-busting
|
@dmitry-shibanov, please let me know if you have additional feedback. I'm interested in having this PR merge for effective cache-busting when multiple Python versions are installed. |
kurtmckee
force-pushed
the
output-python-versions-issue-606
branch
2 times, most recently
from
197e69d
to
949d439
Compare
|
Rebased on |
kurtmckee
force-pushed
the
output-python-versions-issue-606
branch
from
949d439
to
2d235cd
Compare
|
I'm closing this PR because I've written a GitHub action to address my need for strong cache-busting based on installed Python versions. |
Description:
This introduces a
python-versionsoutput, which is a comma-separated string of all installed Python versions and implementations. This is useful for cache-busting when multiple Python versions are installed, as described in #606.For example, it is now possible to cache a
.tox/test directory and invalidate the cache when any of the Python versions change. Caching the.tox/directory offers a significant speedup to test setup times, but if a non-default Python's version changes (say, if 3.11.1 is the default but 3.10.x increments to 3.10.y), it is currently very difficult to invalidate the cache. This new output variable allows for trivial cache invalidation in this situation.To demonstrate the new output variable, I created a workflow with a single
setup-pythonstep that installs multiple CPython and PyPy versions, then usedechoto show the value ofpython-versions.The value displayed is:
which can be used directly in a cache key, or written to a file and included in a call to
hashFiles(). The versions are sorted to avoid sensitivity to the order of Python versions specific in the workflow, but are sorted merely as strings for this purpose, not as semantic versions. Therefore a user can create a workflow that installs Python versions in any order they want, but thepython-versionsoutput string will be stable.NOTE: This work depends on #610 to remove trailing newlines in PyPy versions.
Related issue:
Closes #606
Check list: