Validate Auto-Version

[nickajacks1]

Auto-Version does not validate that newly added versions can be installed and validated.
See: #388

It might be nice if the auto-version github workflow could catch errors like this, for example by attempting to install any auto-versioned packages.

[alecthomas]

This is deliberate, but unfortunate. We used to do this, but it results in a large number of requests and we would get rate limited.

We could potentially add an auto-version --limit flag, so that at most N packages would be updated on each run.

[nickajacks1]

Could the rate limit be worked around by instead creating PRs that get auto merged when the checks pass?
A PR may also make it easier to fix any problems.

[alecthomas]

The rate limiting was caused by the package downloads, I'm not sure how additional PRs would help?

[nickajacks1]

Apologies for the novice question, but I figured I'd just ask to be safe: if I play around with the workflows on my own fork to better understand the issue (replacing the github token with my own), that wouldn't affect this repository in any way would it?

[alecthomas]

No, not at all.

[nickajacks1]

When you say rate limit, do you mean GitHub API rate limit, or is there some other limit? hermit install doesn't make any API calls does it?

I'm not sure how additional PRs would help?

I had assumed that the package sanity check included a hermit install check, which I see now that it does not.