Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove explicit setting of PYTHON in CI jobs #2576

Closed
richardlau opened this issue Mar 16, 2021 · 3 comments
Closed

Remove explicit setting of PYTHON in CI jobs #2576

richardlau opened this issue Mar 16, 2021 · 3 comments
Assignees
Labels
ci-change PSA of configuration changes ci-public

Comments

@richardlau
Copy link
Member

As a follow on to #2507, even with Python 3 available on the CI hosts a lot of the test jobs are explicitly setting PYTHON=python before running the makefile. This issue will track undoing the explicit set on the basis that it is redundant on existing release lines as the default value of PYTHON in the makefile if not set in the environment or via running configure is python.

Refs: nodejs/node#36691
Refs: nodejs/node#37764
Refs: #2507

@richardlau richardlau added ci-public ci-change PSA of configuration changes labels Mar 16, 2021
@richardlau richardlau self-assigned this Mar 16, 2021
@richardlau
Copy link
Member Author

Unfiltered references to `PYTHON=` in CI jobs
$ grep PYTHON= jobs/*.xml
jobs/libuv-test-commit-osx-AshCripps.xml:          <command>PYTHON=python ./gyp_uv.py -f make</command>
jobs/libuv-test-commit-osx-AshCripps.xml:          <command>PYTHON=python make -C out
jobs/libuv-test-commit-windows.xml:SET &quot;PYTHON=c:\python27\python.exe&quot;&#xd;
jobs/node-compile-windows-debug.xml:        <propertiesContent>PYTHON=c:\python27\python.exe</propertiesContent>
jobs/node-linter.xml:                      lint_py3_ret = sh(script: &quot;NODE=node PYTHON=python3 ${env.MAKE} lint-py&quot;, returnStatus: true)
jobs/node-stress-single-test-fips.xml:PYTHON=python ./configure --openssl-fips=$WORKSPACE/fipscanisterdir
jobs/node-stress-single-test-fips.xml:PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE $MAKE -j $(getconf _NPROCESSORS_ONLN)
jobs/node-stress-single-test-fips.xml:PYTHON=python $MAKE build-addons
jobs/node-test-binary-arm-12+.xml:  echo "V=1 CFLAGS='-march=armv7-a' CXXFLAGS='-march=armv7-a' PYTHON=python FLAKY_TESTS=dontcare make test-ci-native" &gt; ./node-ci-exec
jobs/node-test-binary-arm-12+.xml:  echo "CC=should-not-compile CXX=should-not-compile PYTHON=python FLAKY_TESTS=dontcare TEST_CI_ARGS=--run=${RUN_SUBSET},4 make test-ci-js" &gt; ./node-ci-exec
jobs/node-test-binary-arm-OLD.xml:    PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE make test-ci-native
jobs/node-test-binary-arm-OLD.xml:    PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE TEST_CI_ARGS=&quot;--run=${RUN_SUBSET},7&quot; make test-ci-js
jobs/node-test-binary-arm.xml:  echo "PYTHON=python FLAKY_TESTS=dontcare make test-ci-native" &gt; ./node-ci-exec
jobs/node-test-binary-arm.xml:  echo "CC=should-not-compile CXX=should-not-compile PYTHON=python FLAKY_TESTS=dontcare TEST_CI_ARGS=--run=${RUN_SUBSET},6 make test-ci-js" &gt; ./node-ci-exec
jobs/node-test-commit-aix-AshCripps.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix-AshCripps.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" CC=gcc gmake run-ci -j $JOBS
jobs/node-test-commit-aix-AshCripps.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix-AshCripps.xml:NODE_TEST_DIR=${HOME}/build TMP=${HOME}/build PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" gmake run-ci -j $JOBS JOBS=$JOBS</command>
jobs/node-test-commit-aix-richardlau.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix-richardlau.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" CC=gcc gmake run-ci -j $JOBS
jobs/node-test-commit-aix-richardlau.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix-richardlau.xml:NODE_TEST_DIR=${HOME}/build TMP=${HOME}/build PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" gmake run-ci -j $JOBS JOBS=$JOBS</command>
jobs/node-test-commit-aix-shared-lib.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix-shared-lib.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" CC=gcc gmake run-ci -j $JOBS
jobs/node-test-commit-aix-shared-lib.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix-shared-lib.xml:NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64 --shared" gmake run-ci -j $JOBS</command>
jobs/node-test-commit-aix.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" CC=gcc gmake run-ci -j $JOBS
jobs/node-test-commit-aix.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS="--dest-cpu=ppc64" CC=gcc gmake run-ci -j $(lsdev -C -c processor | wc -l)
jobs/node-test-commit-aix.xml:NODE_TEST_DIR=${HOME}/build TMP=${HOME}/build PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" gmake run-ci -j $JOBS JOBS=$JOBS</command>
jobs/node-test-commit-alpine38.xml:  PYTHON=python \
jobs/node-test-commit-arm-sam-github.xml:  PYTHON=python \
jobs/node-test-commit-arm.xml:  PYTHON=python \
jobs/node-test-commit-coverage.xml:#NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python COVTESTS=&quot;test-ci test-internet test-pummel&quot; make coverage -j $(getconf _NPROCESSORS_ONLN)
jobs/node-test-commit-coverage.xml:NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python COVTESTS=&quot;test-all-suites&quot; make coverage -j $(getconf _NPROCESSORS_ONLN)</command>
jobs/node-test-commit-custom-suites-freestyle.xml:export PYTHON=python
jobs/node-test-commit-custom-suites-freestyle.xml:  export PYTHON=python2
jobs/node-test-commit-custom-suites.xml:  PYTHON=python CCACHE_BASEDIR=$(pwd) make -j $JOBS build-ci
jobs/node-test-commit-custom-suites.xml:  PYTHON=python CCACHE_BASEDIR=$(pwd) make bench-addons-build
jobs/node-test-commit-freebsd.xml:which python2 &amp;&amp; export PYTHON=python2
jobs/node-test-commit-linux-as-shared-lib-gdams.xml:  NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS=&quot;--shared&quot; $MAKE run-ci -j $JOBS
jobs/node-test-commit-linux-as-shared-lib.xml:  NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS=&quot;--shared&quot; $MAKE run-ci -j $JOBS
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:  NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE TEST_CI_ARGS=&quot;--node-args --enable-fips&quot; make test-ci -j $JOBS
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python \
jobs/node-test-commit-linux-containered-richardlau.xml:PYTHON=python3 \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python \
jobs/node-test-commit-linux-containered.xml:PYTHON=python3 \
jobs/node-test-commit-linux-coverage-daily.xml:# NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python COVTESTS="test-all-suites" make coverage -j $(getconf _NPROCESSORS_ONLN)
jobs/node-test-commit-linux-coverage-daily.xml:NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python make coverage -j $(getconf _NPROCESSORS_ONLN)
jobs/node-test-commit-linux-fips.xml:PYTHON=python NODE_TEST_DIR=${HOME}/node-tmp FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS=&quot;$CONFIG_FLAGS --openssl-fips=$WORKSPACE/fipscanisterdir&quot; $MAKE run-ci -j $JOBS
jobs/node-test-commit-linux-fips.xml:  NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE TEST_CI_ARGS=&quot;--node-args --enable-fips&quot; $MAKE test-ci -j $JOBS
jobs/node-test-commit-linux-mhdawson.xml:export PYTHON=python
jobs/node-test-commit-linux-richardlau.xml:export PYTHON=python
jobs/node-test-commit-linux-sam-github.xml:export PYTHON=python
jobs/node-test-commit-linux.xml:export PYTHON=python
jobs/node-test-commit-linuxone-sam-github.xml:JOBS=$JOBS CCACHE_BASEDIR="$(pwd)" NODE_TEST_DIR="${HOME}/node-tmp" NODE_COMMON_PORT=15000 PYTHON=python FLAKY_TESTS=dontcare V=1 make run-ci -j $JOBS
jobs/node-test-commit-plinux-sam-github.xml:NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" make run-ci -j $JOBS</command>
jobs/node-test-commit-plinux.xml:NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS="$CONFIG_FLAGS --dest-cpu=ppc64" make run-ci -j $JOBS</command>
jobs/node-test-linter.xml:  make lint-py-build PYTHON=python3 || true
jobs/node-test-linter.xml:  make lint-py PYTHON=python3
jobs/node-test-linter.xml:make lint-py-build PYTHON=python2 || true
jobs/node-test-linter.xml:make lint-ci PYTHON=python2 || {
jobs/node-test-npm-win.xml:        <propertiesContent>PYTHON=c:\python27\python.exe</propertiesContent>
jobs/rv-test-commit-linux-docker.xml:make distclean PYTHON=python3 || true&#13;
jobs/rv-test-commit-linux-docker.xml:        <propertiesContent>PYTHON=python3</propertiesContent>
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:  NODE_TEST_DIR=${HOME}/node-tmp PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE TEST_CI_ARGS=&quot;--node-args --enable-fips&quot; make test-ci -j $JOBS
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/temp-node-test-commit-linux-containered.xml:PYTHON=python \
jobs/test-selector.xml:CCACHE_BASEDIR=&quot;$(pwd)&quot; NODE_TEST_DIR=&quot;${HOME}/node-tmp&quot; NODE_COMMON_PORT=15000 PYTHON=python FLAKY_TESTS=dontcare make run-ci V=1 $MAKE_ARGS
$

I'll filter the list down (for example, we do not need to change the job copies containing build member's names (e.g. node-test-commit-linux-richardlau) under this issue (they can do that themselves if need be)).

@richardlau
Copy link
Member Author

richardlau commented Mar 16, 2021

$ grep PYTHON= jobs/*.xml | cut -d '.' -f 1 | sort -u | awk -F / '{print "- [ ] ["$2"](https://ci.nodejs.org/job/"$2")"}'

@richardlau
Copy link
Member Author

This is now done, in so far as the CI job changes are required. Installation of Python 3 on the CI hosts is tracked under #2507 and ongoing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-change PSA of configuration changes ci-public
Projects
None yet
Development

No branches or pull requests

1 participant