ansible: pin Python 3 to 3.10.8 #3062

richardlau · 2022-10-31T16:42:24Z

Node.js 14 is incompatible with Python 3.11. Pin Python 3 to 3.10.8.
Run Jenkins agent on Java 17.

Updated Windows CI hosts:

Outdated

We're using chocolatey to manage Python on the Windows machines and https://ci.nodejs.org/job/windows-update-reboot/ has updated the Windows machines to Python 3.11.0. Unfortunately, this has broken Node.js 14 builds:

Firstly because of a hard version check: build: support Python 3.11 node#45191 (comment)
Secondly, with the check removed building addons breaks due to the old version of node-gyp in npm 6: [v14.x] build: support Python 3.11 node#45231

So far it looks like the Node.js 14 build is only broken on

test-rackspace-win2012r2_vs2019-x64-4
test-rackspace-win2012r2_vs2019-x64-5

even though other machines also appear to have been updated to Python 3.11.0. I think that is because those two machines were rebooted last week to try to solve disk space issues and the other machines will start to fail the next time they are rebooted.

In order to stop chocolatey updating Python 3 I've had to pin both python and python3 packages (if I don't cup -y all still updates to Python 3.11.0). Unfortunately, to get the playbook to run the pinned task I had to uninstall Python 3.11.0 first:
e.g.

$ ansible -m win_chocolatey -a "name=python version=3.11.0 state=absent remove_dependencies=yes" -v test-rackspace-win2012r2_vs2019-x64-5

However, although the updated playbook does install Python 3.10.8 and pins it (checked with choco pin list and verified cup -y all doesn't update it) the playbook is now broken because py.exe is missing:

TASK [visual-studio : install PyInstaller] *****************************************************************************************************************
task path: /home/rlau/sandbox/github/build/ansible/roles/visual-studio/tasks/main.yml:48
redirecting (type: modules) ansible.builtin.win_command to ansible.windows.win_command
fatal: [test-rackspace-win2012r2_vs2019-x64-5]: FAILED! => {"changed": false, "cmd": "py -3 -m pip install https://github.com/pyinstaller/pyinstaller/archive/develop.tar.gz", "msg": "Exception calling \"SearchPath\" with \"1\" argument(s): \"Could not find file 'py.exe'.\"", "rc": 2}

PLAY RECAP *************************************************************************************************************************************************
test-rackspace-win2012r2_vs2019-x64-5 : ok=48   changed=16   unreachable=0    failed=1    skipped=11   rescued=0    ignored=0

I think I'll need some help here.

cc @joaocgreis @nodejs/platform-windows

richardlau · 2022-10-31T16:43:11Z

So far it looks like the Node.js 14 build is only broken on

test-rackspace-win2012r2_vs2019-x64-4

test-rackspace-win2012r2_vs2019-x64-5

So far I've only run the updated playbook against these two machines.

richardlau · 2022-10-31T19:06:14Z

the playbook is now broken because py.exe is missing:

Fixed by passing Include_launcher=1 as an install_args parameter to win_chocolatey. https://docs.python.org/3/using/windows.html#installing-without-ui suggests this should have been a default but without it py.exe wasn't being installed 🤷.

Now the playbook breaks trying to download an icon:

TASK [jenkins-worker-windows : download Jenkins icon] ******************************************************************************************************
task path: /home/rlau/sandbox/github/build/ansible/roles/jenkins-worker-windows/tasks/main.yml:10
redirecting (type: modules) ansible.builtin.win_get_url to ansible.windows.win_get_url
fatal: [test-rackspace-win2012r2_vs2019-x64-5]: FAILED! => {"changed": false, "dest": "C:\\jenkins.ico", "elapsed": 0.1499964, "msg": "Error when requesting 'Last-Modified' date from 'http://mirrors.jenkins-ci.org/art/jenkins-logo/favicon.ico'. The remote server returned an error: (308) Permanent Redirect.", "status_code": 308, "url": "http://mirrors.jenkins-ci.org/art/jenkins-logo/favicon.ico"}

richardlau · 2022-10-31T20:27:58Z

Now the playbook breaks trying to download an icon:

Fixed that by pointing to the favicon from our Jenkins test CI instance instead of the mirrors.jenkins-ci.org URL.

Also updated the agents to run on Temurin (Java) 17 (#3030).

For now, updated playbook only applied to

test-rackspace-win2012r2_vs2019-x64-4
test-rackspace-win2012r2_vs2019-x64-5

I'll do the others tomorrow as there is the manual step to uninstall python and python3 so it can be downgraded from 3.11.0 and pinned to 3.10.8. The uninstall also appears to leave behind C:\Python311\Scripts\tap2junit.exe which has to also be removed so that the Jenkins jobs pick up the correct C:\Python310\Scripts\tap2junit.exe.

mhdawson

LGTM

Download the Jenkins icon from our own CI server to avoid failure due to redirects.

Refs: nodejs#3030

Node.js 14 is incompatible with Python 3.11. Pin Python 3 to 3.10.8.

richardlau · 2022-11-18T21:32:45Z

I've updated all of the Windows machines except for the arm64 machines. I was able to get a Node.js 14 build to pass: https://ci.nodejs.org/job/node-test-commit-windows-fanned/51761/
(compare to this morning's v14.x daily build which failed: https://ci.nodejs.org/job/node-test-commit-windows-fanned/51756/)

I've not been able to run this against the arm64 machines:

test-msft-win10_vs2017-arm64-1 and test-msft-win10_vs2017-arm64-2 are offline
the ansible/playbooks/jenkins/worker/create.yml playbook fails for me at the package upgrade reboot step on test-nearform_arm-win10_vs2019-arm64-1 and test-nearform_arm-win10_vs2019-arm64-2 [1]

1: e.g.

TASK [package-upgrade : reboot machine to ensure no updates are pending] ***********************************************************************************
task path: /home/rlau/sandbox/github/build/ansible/roles/package-upgrade/tasks/partials/chocolatey.yml:8
redirecting (type: action) ansible.builtin.win_reboot to ansible.windows.win_reboot
redirecting (type: action) ansible.builtin.win_reboot to ansible.windows.win_reboot
win_reboot: rebooting server...
win_reboot validating reboot
win_reboot running post reboot test command
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Exception: Timed out waiting for post-reboot test command (timeout=600)
fatal: [test-nearform_arm-win10_vs2019-arm64-2]: FAILED! => {"changed": true, "elapsed": 651, "msg": "Timed out waiting for post-reboot test command (timeout=600)", "rebooted": true, "unreachable": false}

Maybe this isn't required though for the arm64 machines as it doesn't look like https://ci.nodejs.org/job/windows-arm-simple/ can compile Node.js 14 anyway: https://ci.nodejs.org/job/windows-arm-simple/87/nodes=win10-vs2019-arm64/console
We'd be unlikely to backport any required changes for arm64 back to Node.js 14 (especially since we won't update from npm 6 which has an older version of node-gyp for building addons). cc @nodejs/platform-windows-arm

niyas-sait · 2022-11-21T07:09:23Z

Maybe this isn't required though for the arm64 machines as it doesn't look like https://ci.nodejs.org/job/windows-arm-simple/ can compile Node.js 14 anyway: https://ci.nodejs.org/job/windows-arm-simple/87/nodes=win10-vs2019-arm64/console We'd be unlikely to backport any required changes for arm64 back to Node.js 14 (especially since we won't update from npm 6 which has an older version of node-gyp for building addons). cc @nodejs/platform-windows-arm

Yes, I think we don't have to worry about Node.js 14 for Windows/Arm64.

pbo-linaro · 2022-11-21T08:39:42Z

We are here to help if needed, and we actively monitor build and test of nodejs for main branch on our side through our nightly CI. It uses this script for build and test.

richardlau added ansible platform:windows labels Oct 31, 2022

richardlau marked this pull request as ready for review October 31, 2022 20:28

AshCripps approved these changes Nov 1, 2022

View reviewed changes

joaocgreis approved these changes Nov 2, 2022

View reviewed changes

mhdawson approved these changes Nov 4, 2022

View reviewed changes

richardlau added 3 commits November 18, 2022 13:18

ansible: fix Jenkins icon download

9900dae

Download the Jenkins icon from our own CI server to avoid failure due to redirects.

ansible: update Windows workers to Java 17

bd5258b

Refs: nodejs#3030

ansible: pin Python 3 to 3.10.8 (nodejs#3062)

80334ef

Node.js 14 is incompatible with Python 3.11. Pin Python 3 to 3.10.8.

richardlau force-pushed the winpin branch from bdef01b to 80334ef Compare November 18, 2022 19:49

richardlau merged commit 7fdcf57 into nodejs:main Nov 18, 2022

richardlau deleted the winpin branch November 18, 2022 21:33

richardlau mentioned this pull request Nov 21, 2022

ansible: install temurin JDK on macOS #3085

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ansible: pin Python 3 to 3.10.8 #3062

ansible: pin Python 3 to 3.10.8 #3062

richardlau commented Oct 31, 2022 •

edited

richardlau commented Oct 31, 2022

richardlau commented Oct 31, 2022

richardlau commented Oct 31, 2022

mhdawson left a comment

richardlau commented Nov 18, 2022

niyas-sait commented Nov 21, 2022

pbo-linaro commented Nov 21, 2022

ansible: pin Python 3 to 3.10.8 #3062

ansible: pin Python 3 to 3.10.8 #3062

Conversation

richardlau commented Oct 31, 2022 • edited

richardlau commented Oct 31, 2022

richardlau commented Oct 31, 2022

richardlau commented Oct 31, 2022

mhdawson left a comment

Choose a reason for hiding this comment

richardlau commented Nov 18, 2022

niyas-sait commented Nov 21, 2022

pbo-linaro commented Nov 21, 2022

richardlau commented Oct 31, 2022 •

edited