Discussion: consider disabling wheel testing always

[yarnabrina]

For every release, two PR's are created: first to disable testing full test suite and second to revert first. Is there any reason to not remove it permanently?

[fkiraly]

My main reason for not removing it is:

• I consider it a safety mechanism, reducing the risk of error.
  • It introduces a delay between pressing the green button and the actual release
  • It runs the full test suite between the green button and the actual release, so will not release if there are genuine (not sporadic) failures.
• in the past, I have skipped this due to sporadic failures, or failures we knew were inconsequential. However, in all cases, I've run through the mandatory pre-release test, which I think improved safety and reliability.

Phrased more negatively, if we remove it permanently, I would be apprehensive of irreversible release actions by accident.

[yarnabrina]

I agree with all the safety comments and it makes total sense. The poiof this issue is just noting the skips recently.

To ensure a manual approval or wait period, you can consider this action, or the environment option.

https://github.com/marketplace/actions/manual-workflow-approval
https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments

[fkiraly]

Yes, good idea.

Though, instead of manual approval, how about a simple time delay? Given history, I am not sure if it is feasible to insist on reviews at every release, in the sense that these might not materialize and then things get blocked through inaction.

[yarnabrina]

That's probably a lot easier, you can replace the test call with sleep 1h and that's it.