Install Python 3 everywhere in CI #2507

targos · 2020-12-30T14:46:02Z

This is a requirement for nodejs/node#36691
There are a lot of hosts that only have Python 2 installed.

For example, in ci-release, only the Windows builds work if we remove Python 2 support: https://ci-release.nodejs.org/job/iojs+release/6555/

The text was updated successfully, but these errors were encountered:

cclauss · 2021-01-12T15:27:47Z

I do not have access rights for the link above... How many hosts are we talking about that only have legacy Python?

AshCripps · 2021-01-12T15:42:15Z

It is a fair few of the host - interestingly the mac builds failed in that release build and they most certainty do have python3 installed.

cclauss · 2021-01-12T15:46:25Z

Python 2 died over a year ago (not even security updates) so it is a security exposure to keep those hosts online.

Can you please paste some details on how Py3 on macOS is failing?

targos · 2021-01-12T15:49:29Z

15:41:54 PRODUCTSIGN_CERT=Developer ID Installer: Node.js Foundation
15:41:54 CODESIGN_CERT=Developer ID Application: Node.js Foundation
15:41:54 NOTARIZATION_ID=build@iojs.org
15:41:54 PACKAGEMAKER=~/PackageMaker.app/Contents/MacOS/PackageMaker
15:41:54 
15:41:54 [EnvInject] - Variables injected successfully.
15:41:54 [ws] $ /bin/bash -ex /var/folders/zq/929cmt8n33j9_8x7b06pbzqh0000gp/T/jenkins490912561010356266.sh
15:41:54 + RELEASE_URLBASE=https://nodejs.org/download/test/
15:41:54 + '[' Xtest == Xtest ']'
15:41:54 + disttype=custom
15:41:54 + CUSTOMTAG=test20201230f8775b51b5
15:41:54 + RELEASE_URLBASE=https://nodejs.org/download/test/
15:41:54 + CONFIG_FLAGS='--download-path=/Users/iojs/node-icu/ --verbose'
15:41:54 + '[' -z x ']'
15:41:54 + mkdir -p /Users/iojs/node-icu/
15:41:54 + make -j 2 pkg-upload DESTCPU=x64 ARCH=x64 DISTTYPE=custom DATESTRING=20201230 COMMIT=f8775b51b5 CUSTOMTAG=test20201230f8775b51b5 RELEASE_URLBASE=https://nodejs.org/download/test/ 'CONFIG_FLAGS=--download-path=/Users/iojs/node-icu/ --verbose'
15:41:59 rm -f -r out/macos
15:41:59 mkdir -p out/macos/installer/productbuild
15:41:59 cat tools/macos-installer/productbuild/distribution.xml.tmpl  \
15:41:59 		| sed -E "s/\\{nodeversion\\}/v16.0.0-test20201230f8775b51b5/g" \
15:41:59 		| sed -E "s/\\{npmversion\\}/v7.3.0/g" \
15:41:59 	>out/macos/installer/productbuild/distribution.xml ; \
15:41:59 
15:41:59 Found localization directory tools/macos-installer/productbuild/Resources/en.lproj/
15:41:59 python ./configure \
15:41:59 		--dest-cpu=x64 \
15:41:59 		--tag=test20201230f8775b51b5 \
15:41:59 		--release-urlbase=https://nodejs.org/download/test/ \
15:41:59 		--download-path=/Users/iojs/node-icu/ --verbose --download=all --with-intl=full-icu
15:41:59 Please use python3.9 or python3.8 or python3.7 or python3.6.
15:41:59 	/usr/local/bin/python3.7 ./configure
15:41:59 Node.js configure: Found Python 2.7.16...
15:41:59 make: *** [node-v16.0.0-test20201230f8775b51b5.pkg] Error 1
15:41:59 Build step 'Conditional steps (multiple)' marked build as failure
15:42:00 Archiving artifacts
15:42:00 Sending e-mails to: michael_dawson@ca.ibm.com
15:42:01 Notifying upstream projects of job completion
15:42:01 Finished: FAILURE

AshCripps · 2021-01-12T16:08:57Z

Ah so thats because we call python configure and system python on mac for some reason is still 2.7.16

cclauss · 2021-01-12T16:22:08Z

Yes. I think we just need to sync up to nodejs/node#35983

targos · 2021-01-12T16:44:39Z

Yes. I think we just need to sync up to nodejs/node#35983

I think we have that commit. Your branch is synced up to Dec 29.

AshCripps · 2021-01-12T16:55:42Z

I think the issue is the way the job is setup by directly calling python ./configure. example on my laptop

ash:node/ (master) $ ./configure                                                                                                            [16:53:59]
Node.js configure: Found Python 3.9.1...
...
INFO: configure completed successfully
ash:node/ (master) $ python ./configure                                                                                                     [16:54:15]
Node.js configure: Found Python 2.7.16...
...
INFO: configure completed successfully
ash:node/ (master) $

putting python infront causes it to find python2 not 3

aduh95 · 2021-03-04T12:53:34Z

The Makefile is using $(PYTHON) from the environment. I guess a fix would be to specify the full path of python3 in the environment variable.

richardlau · 2021-03-05T14:59:54Z

Building up the current state on our CI hosts:
ppc hosts: ansible --list-hosts "*ppc*"
s390 hosts: ansible --list-hosts "*s390*"
smartos hosts: ansible --list-hosts "*smartos*"
freebsd hosts: ansible --list-hosts "*freebsd*"

host	`python --version`	`python3 --version`	notes	action
release-ibm-aix71-ppc64_be-2	2.7.18	-		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
release-iinthecloud-ibmi72-ppc64_be-1			unreachable
release-osuosl-aix72-ppc64_be-1	2.7.18	3.6.12		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
release-osuosl-centos7-ppc64_le-1	2.7.5	3.6.8
test-ibm-aix71-ppc64_be-3	2.7.18	-		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
test-ibm-aix71-ppc64_be-4	2.7.18	-		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
test-iinthecloud-ibmi72-ppc64_be-1	3.6.10	3.6.10		Python 3.6.12 installed in `/QOpenSys/pkgs/bin/python` and `/QOpenSys/pkgs/bin/python3` #2567
test-iinthecloud-ibmi72-ppc64_be-2	2.7.18	-		Python 3.6.12 installed in `/QOpenSys/pkgs/bin/python` and `/QOpenSys/pkgs/bin/python3` #2567
test-osuosl-aix72-ppc64_be-2	2.7.18	3.6.12		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
test-osuosl-aix72-ppc64_be-3	2.7.18	3.6.12		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
test-osuosl-aix72-ppc64_be-4	2.7.18	3.6.12		Python 3.7.9 installed in `/opt/freeware/bin/python3` #2564
test-osuosl-centos7-ppc64_le-1	2.7.5	3.6.8
test-osuosl-centos7-ppc64_le-2	2.7.5	3.6.8
release-ibm-rhel7-s390x-1	2.7.5	-		Python 3.6.8 installed in `/usr/bin/python3` #2571
release-marist-zos24-s390x-1	2.7.13	-	Not building Node.js yet. libuv builds do not use Python.
test-ibm-rhel7-s390x-1	2.7.5	-		Python 3.6.8 installed in `/usr/bin/python3` #2571
test-ibm-rhel7-s390x-2	2.7.5	-		Python 3.6.8 installed in `/usr/bin/python3` #2571
test-ibm-rhel7-s390x-3	2.7.5	-		Python 3.6.8 installed in `/usr/bin/python3` #2571
test-ibm-rhel7-s390x-4	2.7.5	-		Python 3.6.8 installed in `/usr/bin/python3` #2571
test-marist-zos24-s390x-1	2.7.13	-	Not building Node.js yet. libuv builds do not use Python.
test-marist-zos24-s390x-2	2.7.13	-	Not building Node.js yet. libuv builds do not use Python.
infra-joyent-smartos15-x64-1	2.7.11	-	Infra machine -- we do not build on this
release-joyent-smartos17-x64-2	2.7.14	-		Python 3.6.4 installed in `/opt/local/bin/python3` #2575
release-joyent-smartos18-x64-2	2.7.15	-		Python 3.7.1 installed in `/opt/local/bin/python3` #2575
test-joyent-smartos17-x64-3	2.7.14	-		Python 3.6.4 installed in `/opt/local/bin/python3` #2575
test-joyent-smartos17-x64-4	2.7.14	-		Python 3.6.4 installed in `/opt/local/bin/python3` #2575
test-joyent-smartos18-x64-3	2.7.15	-		Python 3.7.1 installed in `/opt/local/bin/python3` #2575
test-joyent-smartos18-x64-4	2.7.15	-		Python 3.7.1 installed in `/opt/local/bin/python3` #2575
test-digitalocean-freebsd11-x64-1	3.7.10	3.7.10	Python 2.7.18 in `/usr/local/bin/python2`
test-digitalocean-freebsd11-x64-2	3.7.10	3.7.10	Python 2.7.18 in `/usr/local/bin/python2`
release-packetnet-centos7-arm64-1	2.7.5	-		Python 3.6.8 installed under `/usr/bin/python3` #2585
test-packetnet-centos7-arm64-1	2.7.5	-		Python 3.6.8 installed under `/usr/bin/python3` #2585
test-packetnet-centos7-arm64-2	2.7.5	-		Python 3.6.8 installed under `/usr/bin/python3` #2585
test-packetnet-ubuntu1604-arm64-1	2.7.12	3.5.2	not used for Node.js 14+
test-packetnet-ubuntu1604-arm64-2	2.7.12	3.5.2	not used for Node.js 14+
release-digitalocean-centos7-x64-1	2.7.5	-	ansible errors	Python 3.6.8 installed under `/usr/bin/python3`
test-rackspace-centos7-x64-1	2.7.5	-	ansible errors	Python 3.6.8 installed under `/usr/bin/python3`
test-softlayer-centos7-x64-1	2.7.5	-	ansible errors	Python 3.6.8 installed under `/usr/bin/python3`

Others:

container	`python --version`	`python3 --version`	notes	action
alpine 3.11	2.7.18	-	Dockerfile	Python 3.8.2 installed under `/usr/bin/python3` #2589
alpine 3.12	3.8.3	3.8.3	Dockerfile - not used to build Node.js 10 and 12 (no Python 2)

Refs: #2507

cclauss · 2021-03-16T06:03:58Z

Can the chart #2507 (comment) be updated to provide reviewers more confidence that we are ready to move forward with nodejs/node#36691 ?

Refs: #2507

targos · 2021-03-20T09:05:01Z

python3 is missing from test-packetnet-centos7-arm64-2: https://ci.nodejs.org/job/node-test-commit-arm/36534/nodes=centos7-arm64-gcc8/console

cclauss · 2021-03-20T13:07:08Z

Nice work!! Looking at the chart above, Python >= 3.6 is now installed somewhere on all machines where test builds are done.

release-iinthecloud-ibmi72-ppc64_be-1 is unreachable (perhaps there is one i too many) and test-packetnet-centos7-arm64-2 needs Py3 added.

Refs: #2507

richardlau · 2021-03-22T17:58:54Z

I've been going (slowly) through the various platforms we have and updating the table in #2507 (comment). The IBM platforms are done, as is SmartOS and the centos7-arm64 hosts.

Next on my list to look at are the Pi's, which at first glance appear to have Python 3 available but it looks like the jobs actually execute builds in docker containers which do not have Python 3 available. e.g. the docker file for Debian Stretch (9): https://github.com/nodejs/build/blob/master/ansible/roles/jenkins-worker/templates/armv7_stretch.Dockerfile.j2
I'm investigating but it looks like the latest version of Python 3 available on Debian Stretch is Python 3.5.3: https://packages.debian.org/stretch/python3

I haven't started looking at the Linux x64 platforms yet -- I suspect some of the containers we're using there will also need updating to install Python 3 on them.

cclauss · 2021-03-22T18:14:45Z

The deadsnakes PPA would enable the installation of Python 3.9.2 or other versions. Another way to go is to use pyenv.

targos · 2021-03-23T09:00:13Z

centos-7-64-gcc8 don't have python3: https://ci.nodejs.org/job/node-test-commit-linux/40525/nodes=centos7-64-gcc8/console

cclauss · 2021-03-23T13:22:05Z

centos-7-64-gcc8 don't have python3

According to the chart above, perhaps all that test needs is export PATH="/opt/rh/rh-python38:$PATH"

richardlau · 2021-03-23T18:00:17Z

Ugh. Our three centos7-x64 hosts are not liking ansible at all. I've manually put python3 (3.6.8 at this time) on them and will raised #2587 to discuss the problems we have.

targos · 2021-03-29T18:47:26Z

I've been going (slowly) through the various platforms we have and updating the table in #2507 (comment). The IBM platforms are done, as is SmartOS and the centos7-arm64 hosts.

Next on my list to look at are the Pi's, which at first glance appear to have Python 3 available but it looks like the jobs actually execute builds in docker containers which do not have Python 3 available. e.g. the docker file for Debian Stretch (9): https://github.com/nodejs/build/blob/master/ansible/roles/jenkins-worker/templates/armv7_stretch.Dockerfile.j2
I'm investigating but it looks like the latest version of Python 3 available on Debian Stretch is Python 3.5.3: https://packages.debian.org/stretch/python3

I haven't started looking at the Linux x64 platforms yet -- I suspect some of the containers we're using there will also need updating to install Python 3 on them.

I can try to help with the RPi configuration. How do I convert the template to a real Dockerfile?

rvagg · 2021-03-30T02:38:55Z

The dockerfiles get run on the Pi's themselves, they are armv7 native so hard to replicate outside of that. If you log in to one of the Pi's (a 3 would be good) and sudo su root to become root then go into /root/ you'll see directories for each of the templates and a Dockerfile inside there. So no need to even recreate the template, use the ones that Ansible left last time it did the build. You can mess around in there and build an image and try it out on there until you get it right and then come back to this repo and make your changes in the templates.

Refs: #2507

Trott · 2021-04-18T11:05:41Z

Looks like we need to update Python on test-rackspace-ubuntu1604-x64-1 (or, uh, move to a more recent Ubuntu, see #2625).

https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/20329/console

00:01:16 + make -j 4 build-ci
00:01:16 python3 ./configure --verbose 
00:01:16 Please use python3.9 or python3.8 or python3.7 or python3.6.
00:01:16 Node.js configure: Found Python 3.5.2...
00:01:16 Makefile:531: recipe for target 'build-ci' failed
00:01:16 make: *** [build-ci] Error 1

targos · 2021-04-18T11:19:23Z

I think someone needs to update the config of node-daily-master. node-test-commit-custom-suites-freestyle already defaults to ubuntu1804-64.

Trott · 2021-04-18T11:36:34Z

I think someone needs to update the config of node-daily-master. node-test-commit-custom-suites-freestyle already defaults to ubuntu1804-64.

Oh, that was easy, then. I've updated the label to be 1804 instead of 1604. (I wonder if I should have blanked it out so it always gets a default value?)

richardlau · 2021-04-18T12:57:08Z

Ah sorry, I did change the default and the config for the main node-test-commit job but missed the daily.

This also changes the base image to balenalib/rpi-raspbian:stretch. The old one is no longer maintained. Refs: #2507

richardlau self-assigned this Mar 4, 2021

cclauss mentioned this issue Mar 10, 2021

Node.js Technical Steering Committee (TSC) Meeting 2021-03-11 nodejs/TSC#980

Closed

This was referenced Mar 10, 2021

Remove support for Python 2 nodejs/node#36691

Merged

ansible: add python3 to AIX hosts #2564

Merged

ansible: install Python 3 on IBM i #2567

Merged

richardlau added a commit that referenced this issue Mar 15, 2021

ansible: install Python 3 on IBM i (#2567)

196227c

Refs: #2507

richardlau mentioned this issue Mar 15, 2021

ansible: add Python 3 to LinuxONE hosts #2571

Merged

richardlau added a commit that referenced this issue Mar 16, 2021

ansible: add Python 3 to LinuxONE hosts (#2571)

f7de7bf

Refs: #2507

This was referenced Mar 16, 2021

Remove explicit setting of PYTHON in CI jobs #2576

Closed

Node.js Build WorkGroup Meeting 2021-03-16 #2562

Closed

targos mentioned this issue Mar 22, 2021

build: default PYTHON to python3 in Makefile nodejs/node#37764

Closed

richardlau added a commit that referenced this issue Mar 22, 2021

ansible: add Python 3 to SmartOS hosts (#2575)

6882ed5

Refs: #2507

richardlau mentioned this issue Mar 22, 2021

ansible: add Python 3 to centos7-arm64 hosts #2585

Merged

This was referenced Mar 23, 2021

Ansible errors on centos7-x64 #2587

Closed

ansible: add Python 3 to alpine 3.11 container #2589

Merged

targos mentioned this issue Mar 30, 2021

Request for temporary access to Raspberry Pi's #2598

Closed

richardlau mentioned this issue Mar 30, 2021

Debian versions and Python 3 (Node.js 16 and later) #2600

Closed

richardlau added a commit that referenced this issue Mar 31, 2021

ansible: add Python 3 to alpine 3.11 container (#2589)

01dc507

Refs: #2507

targos mentioned this issue Apr 3, 2021

ansible,docker: install Python 3.9 in Stretch RPi's #2606

Merged

richardlau pushed a commit that referenced this issue Apr 19, 2021

ansible,docker: install Python 3.9 in Stretch RPi's (#2606)

79d8678

This also changes the base image to balenalib/rpi-raspbian:stretch. The old one is no longer maintained. Refs: #2507

targos closed this as completed Jun 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install Python 3 everywhere in CI #2507

Install Python 3 everywhere in CI #2507

targos commented Dec 30, 2020

cclauss commented Jan 12, 2021

AshCripps commented Jan 12, 2021

cclauss commented Jan 12, 2021

targos commented Jan 12, 2021

AshCripps commented Jan 12, 2021

cclauss commented Jan 12, 2021

targos commented Jan 12, 2021

AshCripps commented Jan 12, 2021

aduh95 commented Mar 4, 2021

richardlau commented Mar 5, 2021 •

edited

cclauss commented Mar 16, 2021 •

edited

targos commented Mar 20, 2021

cclauss commented Mar 20, 2021 •

edited

richardlau commented Mar 22, 2021

cclauss commented Mar 22, 2021 •

edited

targos commented Mar 23, 2021

cclauss commented Mar 23, 2021

richardlau commented Mar 23, 2021 •

edited

targos commented Mar 29, 2021

rvagg commented Mar 30, 2021

Trott commented Apr 18, 2021

targos commented Apr 18, 2021

Trott commented Apr 18, 2021

richardlau commented Apr 18, 2021

Install Python 3 everywhere in CI #2507

Install Python 3 everywhere in CI #2507

Comments

targos commented Dec 30, 2020

cclauss commented Jan 12, 2021

AshCripps commented Jan 12, 2021

cclauss commented Jan 12, 2021

targos commented Jan 12, 2021

AshCripps commented Jan 12, 2021

cclauss commented Jan 12, 2021

targos commented Jan 12, 2021

AshCripps commented Jan 12, 2021

aduh95 commented Mar 4, 2021

richardlau commented Mar 5, 2021 • edited

cclauss commented Mar 16, 2021 • edited

targos commented Mar 20, 2021

cclauss commented Mar 20, 2021 • edited

richardlau commented Mar 22, 2021

cclauss commented Mar 22, 2021 • edited

targos commented Mar 23, 2021

cclauss commented Mar 23, 2021

richardlau commented Mar 23, 2021 • edited

targos commented Mar 29, 2021

rvagg commented Mar 30, 2021

Trott commented Apr 18, 2021

targos commented Apr 18, 2021

Trott commented Apr 18, 2021

richardlau commented Apr 18, 2021

richardlau commented Mar 5, 2021 •

edited

cclauss commented Mar 16, 2021 •

edited

cclauss commented Mar 20, 2021 •

edited

cclauss commented Mar 22, 2021 •

edited

richardlau commented Mar 23, 2021 •

edited