./configure fails after Python version default changed #30129

codebytere · 2019-10-26T03:55:15Z

Version: master
Platform: Darwin
Subsystem: build

b2ccbb2 seems to have caused ./configure to fail locally.

Before:

node on git:master ❯ ./configure                                                      8:50PM
Node configure: Found Python 2.7.10...
INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
INFO: configure completed successfully

After:

node on git:master❯ ./configure                                                       8:50PM
Node configure: Found Python 3.7.4...
INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.
No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.
gyp: No Xcode or CLT version detected!
Error running GYP

No changes were made to XCode, and i have command line tools on my machine already. Reverting this commit locally resolves the issue, but i'm not sure why it would have caused it to begin with?

cc @cclauss perhaps?

The text was updated successfully, but these errors were encountered:

richardlau · 2019-10-26T04:22:42Z

@codebytere Could you try applying the changes from nodejs/node-gyp#1890 to tools/gyp/pylib/gyp/xcode_emulation.py?

cclauss · 2019-10-26T05:12:34Z

You should still be able to control which Python make uses with PYTHON=python2

I am interested in the outcome of Richard’s 1890 experiment above.

Also, nodejs/node-gyp#1939 (merged) that really focuses on issues related to No Xcode or CLT version detected!.

If you have recently upgraded to macOS Catalina then please provide a review of nodejs/node-gyp#1940.

ZYSzys · 2019-10-26T05:59:42Z

@codebytere Could you try applying the changes from nodejs/node-gyp#1890 to tools/gyp/pylib/gyp/xcode_emulation.py?

It was still fail after applying nodejs/node-gyp#1890

index 9a58df4782..610f90de21 100644
--- a/tools/gyp/pylib/gyp/xcode_emulation.py
+++ b/tools/gyp/pylib/gyp/xcode_emulation.py
@@ -20,6 +20,8 @@ import sys
 import tempfile
 from gyp.common import GypError

+PY3 = bytes != str
+
 # Populated lazily by XcodeVersion, for efficiency, and to fix an issue when
 # "xcodebuild" is called too quickly (it has been found to return incorrect
 # version number).
@@ -1407,7 +1409,7 @@ def XcodeVersion():
   except:
     version = CLTVersion()
     if version:
-      version = re.match(r'(\d+\.\d+\.?\d*)', version).groups()[0]
+      version = ".".join(version.split(".")[:3])
     else:
       raise GypError("No Xcode or CLT version detected!")
     # The CLT has no build information, so we return an empty string.
@@ -1453,6 +1455,8 @@ def GetStdoutQuiet(cmdlist):
   job = subprocess.Popen(cmdlist, stdout=subprocess.PIPE,
                          stderr=subprocess.PIPE)
   out = job.communicate()[0]
+  if PY3:
+    out = out.decode("utf-8")
   if job.returncode != 0:
     raise GypError('Error %d running %s' % (job.returncode, cmdlist[0]))
   return out.rstrip('\n')
@@ -1463,6 +1467,8 @@ def GetStdout(cmdlist):
   Raises |GypError| if the command return with a non-zero return code."""
   job = subprocess.Popen(cmdlist, stdout=subprocess.PIPE)
   out = job.communicate()[0]
+  if PY3:
+    out = out.decode("utf-8")
   if job.returncode != 0:
     sys.stderr.write(out + '\n')

The error shows:

 ~/Projects/node ./configure                                  [13:50:00]
Node configure: Found Python 3.7.4...
INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
Traceback (most recent call last):
  File "./configure", line 25, in <module>
    import configure
  File "/Users/zhangyongsheng/Projects/node/configure.py", line 1755, in <module>
    run_gyp(gyp_args)
  File "tools/gyp_node.py", line 54, in run_gyp
    rc = gyp.main(args)
  File "tools/gyp/pylib/gyp/__init__.py", line 547, in main
    return gyp_main(args)
  File "tools/gyp/pylib/gyp/__init__.py", line 532, in gyp_main
    generator.GenerateOutput(flat_list, targets, data, params)
  File "tools/gyp/pylib/gyp/generator/make.py", line 2181, in GenerateOutput
    part_of_all=qualified_target in needed_targets)
  File "tools/gyp/pylib/gyp/generator/make.py", line 773, in Write
    extra_mac_bundle_resources, part_of_all)
  File "tools/gyp/pylib/gyp/generator/make.py", line 879, in WriteActions
    env = self.GetSortedXcodeEnv()
  File "tools/gyp/pylib/gyp/generator/make.py", line 1864, in GetSortedXcodeEnv
    additional_settings)
  File "tools/gyp/pylib/gyp/xcode_emulation.py", line 1772, in GetSortedXcodeEnv
    additional_settings)
  File "tools/gyp/pylib/gyp/xcode_emulation.py", line 1683, in _GetXcodeEnv
    if XcodeVersion() >= '0500' and not env.get('SDKROOT'):
TypeError: '>=' not supported between instances of 'tuple' and 'str'

richardlau · 2019-10-26T06:02:25Z

@ZYSzys Please try nodejs/node-gyp#1939 as well then.

cclauss · 2019-10-26T06:17:22Z

The code before nodejs/node-gyp#1932 might have been eating exceptions that are now appearing. This is safer in the long term but might cause hiccups in the near term.

Trott · 2019-10-27T04:40:16Z

You should still be able to control which Python gets used with PYTHON=python2

For some reason, that doesn't seem to work for me. Or at least it still reports finding Python 3.7.1, suggesting it's still using it (and getting me the same results).

$ PYTHON=python2 ./configure
Node configure: Found Python 3.7.1...
INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.
No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.
gyp: No Xcode or CLT version detected!
Error running GYP
$

To anticipate the obvious question:

$ python2 --version
Python 2.7.15
$

Trott · 2019-10-27T05:01:31Z

After applying https://github.com/nodejs/node-gyp/pull/1890/commits/19398a4b0d88581e819b697445771a6e69759cd5.patch, I get similar results to those reported by @ZYSzys.

Applying https://github.com/nodejs/node-gyp/pull/1939.patch on top of that does not apply cleanly, but the patch-merge-conflict to sort out is pretty straightforward I think, so I did that. Now ./configure works, but make fails:

  File "./gyp-mac-tool", line 673
    max_value_length = len(max(filelist.items(), key=lambda (k,v):len(v))[1])
                                                            ^
SyntaxError: invalid syntax

cclauss · 2019-10-27T06:37:50Z

I was inaccurate when I wrote:

“You should still be able to control which Python ~~gets used~~ with PYTHON=python2”

I should have instead written:

“You should still be able to control which Python make uses with PYTHON=python2“

nodejs/node-gyp#1939 Is a rewrite of this logic and would be my preferred solution.

Fixed in #30146, `lambda (k,v):len(v) is indeed a Python 3 syntax error which flake8 can spot automatically. #30143 reconfigures flake8 to find syntax errors even in our dependencies.

As discussed in #30129 (comment), when we vendor in code, we own the Syntax Errors in that code. This PR adds The `.flake8` config file at the root of this repo puts blinders on the linting of our dependencies so this test disables that file before linting.

bnoordhuis · 2019-10-27T13:31:39Z

FWIW, python2 configure.py or python2.7 configure.py can be used as a workaround.

sam-github · 2019-10-29T14:46:22Z

FTR, Ben's comment above is the right way to force configure to use a specific python executable, ./configure has never respected a PYTHON env variable.

As discussed in #30129 (comment), when we vendor in code, we own the Syntax Errors in that code. This PR adds The `.flake8` config file at the root of this repo puts blinders on the linting of our dependencies so this test disables that file before linting.

As discussed in #30129 (comment), when we vendor in code, we own the Syntax Errors in that code. This PR adds The `.flake8` config file at the root of this repo puts blinders on the linting of our dependencies so this test disables that file before linting. fixup: allow_failures until dependencies pass PR-URL: #30143 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com>

cclauss · 2019-11-05T18:59:01Z

#30262 (comment) is correct. We have cleaned up the nodejs/node-gyp's gyp/pylib/gyp/xcode_emulation.py file especially in nodejs/node-gyp#1890 nodejs/node-gyp#1895 nodejs/node-gyp#1932 and nodejs/node-gyp#1939 but we have not brought those changes back to this repo's tools/gyp/pylib/gyp/xcode_emulation.py file which is required to fix this issue. PR on its way...

As discussed in #30129 (comment), when we vendor in code, we own the Syntax Errors in that code. This PR adds The `.flake8` config file at the root of this repo puts blinders on the linting of our dependencies so this test disables that file before linting. fixup: allow_failures until dependencies pass PR-URL: #30143 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com>

PR-URL: #30272 Fixes: #30129 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Shelley Vohr <codebytere@gmail.com> Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>

codebytere added the build Issues and PRs related to build files or the CI. label Oct 26, 2019

codebytere changed the title ~~./ configure fails after Python version default changed~~ ./configure fails after Python version default changed Oct 26, 2019

richardlau added python PRs and issues that require attention from people who are familiar with Python. macos Issues and PRs related to the macOS platform / OSX. labels Oct 26, 2019

codebytere mentioned this issue Oct 26, 2019

Calling Catalina users in @nodejs/collaborators nodejs/node-gyp#1936

Closed

cclauss mentioned this issue Oct 27, 2019

build: find Python syntax errors in dependencies #30143

Closed

4 tasks

cclauss mentioned this issue Oct 27, 2019

tools: fix Python 3 syntax error in mac_tool.py #30146

Closed

4 tasks

huenchao mentioned this issue Oct 30, 2019

构建本地debug node的环境，我用cmake编译 huenchao/node-study#2

Open

richardlau mentioned this issue Oct 31, 2019

Build: No module named 'distutils.spawn' #30189

Closed

chrmoritz mentioned this issue Nov 5, 2019

v13.1.0 release proposal #30262

Merged

cclauss mentioned this issue Nov 5, 2019

tools: pull xcode_emulation.py fixes from node-gyp #30272

Closed

1 task

This was referenced Nov 6, 2019

build: configure failed with python 3.7.4 #30215

Closed

Build from source error on Catalina #30276

Closed

ChALkeR closed this as completed in 0673dfc Nov 6, 2019

captain-kark mentioned this issue Nov 11, 2020

Installing electron on MacOS Catalina nodejs/node-gyp#2257

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

./configure fails after Python version default changed #30129

./configure fails after Python version default changed #30129

codebytere commented Oct 26, 2019 •

edited

richardlau commented Oct 26, 2019

cclauss commented Oct 26, 2019 •

edited

ZYSzys commented Oct 26, 2019

richardlau commented Oct 26, 2019

cclauss commented Oct 26, 2019

Trott commented Oct 27, 2019

Trott commented Oct 27, 2019

cclauss commented Oct 27, 2019 •

edited

bnoordhuis commented Oct 27, 2019

sam-github commented Oct 29, 2019

cclauss commented Nov 5, 2019 •

edited

./configure fails after Python version default changed #30129

./configure fails after Python version default changed #30129

Comments

codebytere commented Oct 26, 2019 • edited

richardlau commented Oct 26, 2019

cclauss commented Oct 26, 2019 • edited

ZYSzys commented Oct 26, 2019

richardlau commented Oct 26, 2019

cclauss commented Oct 26, 2019

Trott commented Oct 27, 2019

Trott commented Oct 27, 2019

cclauss commented Oct 27, 2019 • edited

bnoordhuis commented Oct 27, 2019

sam-github commented Oct 29, 2019

cclauss commented Nov 5, 2019 • edited

codebytere commented Oct 26, 2019 •

edited

cclauss commented Oct 26, 2019 •

edited

cclauss commented Oct 27, 2019 •

edited

cclauss commented Nov 5, 2019 •

edited