Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion `(source_it) != (source_.end())' failed while doing make #39408

Closed
gireeshpunathil opened this issue Jul 16, 2021 · 21 comments
Closed

Comments

@gireeshpunathil
Copy link
Member

Version

master

Platform

Linux 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

build

What steps will reproduce the bug?

$./configure
$make -j4

ACTION _home_gireesh_node_node_gyp_node_target_node_mksnapshot /home/gireesh/node/out/Release/obj/gen/node_snapshot.cc
/home/gireesh/node/out/Release/node_mksnapshot[660863]: ../src/node_native_module.cc:226:v8::MaybeLocal<v8::String> node::native_module::NativeModuleLoader::LoadBuiltinModuleSource(v8::Isolate*, const char*): Assertion `(source_it) != (source_.end())' failed.
 1: 0x55d5331d5ee4 node::Abort() [/home/gireesh/node/out/Release/node_mksnapshot]
 2: 0x55d5331d5f78  [/home/gireesh/node/out/Release/node_mksnapshot]
 3: 0x55d5333f8fdd node::native_module::NativeModuleLoader::LoadBuiltinModuleSource(v8::Isolate*, char const*) [/home/gireesh/node/out/Release/node_mksnapshot]
 4: 0x55d5333f95d1 node::native_module::NativeModuleLoader::LookupAndCompile(v8::Local<v8::Context>, char const*, std::vector<v8::Local<v8::String>, std::allocator<v8::Local<v8::String> > >*, node::native_module::NativeModuleLoader::Result*) [/home/gireesh/node/out/Release/node_mksnapshot]
 5: 0x55d533234649 node::native_module::NativeModuleEnv::LookupAndCompile(v8::Local<v8::Context>, char const*, std::vector<v8::Local<v8::String>, std::allocator<v8::Local<v8::String> > >*, node::Environment*) [/home/gireesh/node/out/Release/node_mksnapshot]
 6: 0x55d53336f36e node::InitializePrimordials(v8::Local<v8::Context>) [/home/gireesh/node/out/Release/node_mksnapshot]
 7: 0x55d53336f560 node::GetPerContextExports(v8::Local<v8::Context>) [/home/gireesh/node/out/Release/node_mksnapshot]
 8: 0x55d53336f2a0 node::InitializePrimordials(v8::Local<v8::Context>) [/home/gireesh/node/out/Release/node_mksnapshot]
 9: 0x55d53336f695 node::NewContext(v8::Isolate*, v8::Local<v8::ObjectTemplate>) [/home/gireesh/node/out/Release/node_mksnapshot]
10: 0x55d53327a4a9 node::SnapshotBuilder::Generate(node::SnapshotData*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) [/home/gireesh/node/out/Release/node_mksnapshot]
11: 0x55d53327b014 node::SnapshotBuilder::Generate(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >) [/home/gireesh/node/out/Release/node_mksnapshot]
12: 0x55d532e05d7d main [/home/gireesh/node/out/Release/node_mksnapshot]
13: 0x7f89037610b3 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
14: 0x55d532e0e46e _start [/home/gireesh/node/out/Release/node_mksnapshot]
Aborted (core dumped)
make: *** [node.target.mk:26: /home/gireesh/node/out/Release/obj/gen/node_snapshot.cc] Error 134
rm 2561388a0ca6c2168de218f405f8f7b697c77862.intermediate 9d285d3140c975d071f47f872a2170cedfc46bc5.intermediate aaef1ed75da9c226afcc645ea47e3916fde6ec36.intermediate

How often does it reproduce? Is there a required condition?

everytime

What is the expected behavior?

node builds successfully

What do you see instead?

build crashes

Additional information

first time I am building in this system, so suspect something specific to build toolchain versions, resource limit etc. but cannot figure out what.

@legendecas
Copy link
Member

legendecas commented Jul 16, 2021

This is typically for built in module not been found. Can you re-configure to generate refresh new build scripts and try again?

Edit: just found that one of the re-production steps is ./configure. There is a change on master branch recently that all builtin module files were globbed to be bundled into node binary. You may check on the variable node_library_files in config.gypi to see if it is present and find out if there are any files missing from the list.

@legendecas
Copy link
Member

@gireeshpunathil does that help?

@gireeshpunathil
Copy link
Member Author

@legendecas - thanks for the pointers provided, unfortunately, I did not have time to look at that yet. I will try to squeeze in sometime this week. Meanwhile, does it help if I recreate this in a docker image and pass it to you for your inspection?

@gireeshpunathil
Copy link
Member Author

@legendecas - this was super-quick and super-useful!!!

Cannot find native builtin. "internal/per_context/primordials" 1: 0x564f109d3ee4 node::Abort() 

this is what I get with your patch. How do I fix this now? Any idea?

@legendecas
Copy link
Member

legendecas commented Jul 20, 2021

That sounds like a problem on the config.gypi, i.e. no files are included in the build -- internal/per_context/primordials is the first file been evaluated during the snapshot process.

Can you ./configure again?

@gireeshpunathil
Copy link
Member Author

In one system:

gireesh@s1:~/node$ grep "primordials" config.gypi
                                         'lib/internal/per_context/primordials.js',
gireesh@s1:~/node$ ./configure
Node.js configure: Found Python 3.8.10...
INFO: configure completed successfully
gireesh@s1:~/node$ grep "primordials" config.gypi
gireesh@s1:~/node$ 

In another system:

root@828e127b5089:/node# grep "primordials" config.gypi
                                         'lib/internal/per_context/primordials.js',
root@828e127b5089:/node# ./configure
Node.js configure: Found Python 3.8.10...
INFO: configure completed successfully
root@828e127b5089:/node# grep "primordials" config.gypi
                                         'lib/internal/per_context/primordials.js',
root@828e127b5089:/node# 

I am confused. :)

@legendecas
Copy link
Member

@gireeshpunathil if the first output is the problem behaving one, can you try this command python3 -c 'import sys;sys.path.insert(0, \'tools\');from utils import SearchFiles;print(SearchFiles(\'lib\', \'js\'))' on the system that has the problem?

@gireeshpunathil
Copy link
Member Author

@legendecas - here it is:

['lib/assert/strict.js', 'lib/dns/promises.js', 'lib/fs/promises.js', 'lib/internal/abort_controller.js', 'lib/internal/assert.js', 'lib/internal/async_hooks.js', 'lib/internal/blob.js', 'lib/internal/blocklist.js', 'lib/internal/buffer.js', 'lib/internal/child_process.js', 'lib/internal/cli_table.js', 'lib/internal/constants.js', 'lib/internal/dgram.js', 'lib/internal/dtrace.js', 'lib/internal/encoding.js', 'lib/internal/error_serdes.js', 'lib/internal/errors.js', 'lib/internal/event_target.js', 'lib/internal/fixed_queue.js', 'lib/internal/freelist.js', 'lib/internal/freeze_intrinsics.js', 'lib/internal/heap_utils.js', 'lib/internal/histogram.js', 'lib/internal/http.js', 'lib/internal/idna.js', 'lib/internal/inspector_async_hook.js', 'lib/internal/js_stream_socket.js', 'lib/internal/linkedlist.js', 'lib/internal/net.js', 'lib/internal/options.js', 'lib/internal/priority_queue.js', 'lib/internal/querystring.js', 'lib/internal/repl.js', 'lib/internal/socket_list.js', 'lib/internal/socketaddress.js', 'lib/internal/stream_base_commons.js', 'lib/internal/timers.js', 'lib/internal/trace_events_async_hooks.js', 'lib/internal/tty.js', 'lib/internal/url.js', 'lib/internal/util.js', 'lib/internal/v8_prof_polyfill.js', 'lib/internal/v8_prof_processor.js', 'lib/internal/validators.js', 'lib/internal/watchdog.js', 'lib/internal/worker.js', 'lib/path/posix.js', 'lib/path/win32.js', 'lib/stream/promises.js', 'lib/stream/web.js', 'lib/timers/promises.js', 'lib/util/types.js']

@gireeshpunathil
Copy link
Member Author

@legendecas - I think at this point I should mention this - now that you are bringing python into the issue:

diff --git a/tools/utils.py b/tools/utils.py
index 9d53bd56bd..35a0bd22a0 100644
--- a/tools/utils.py
+++ b/tools/utils.py
@@ -109,7 +109,7 @@ def IsWindows():
 
 
 def SearchFiles(dir, ext):
-  list = glob.glob(dir+ '/**/*.' + ext, recursive=True)
+  list = glob.glob(dir+ '/**/*.' + ext)
   if sys.platform == 'win32':
     list = [ x.replace('\\', '/')for x in list]
   return list

I had to make this change in utils.py, as without that it was giving an error that the the function does not take the second parameter. Do you think this could have any effect on the failure that we are seeing?

@legendecas
Copy link
Member

@gireeshpunathil I believe that option is exactly the issue here. Can you share the error when the function takes the second parameter?

@gireeshpunathil
Copy link
Member Author

  ACTION Generating node protocol sources from protocol json aaef1ed75da9c226afcc645ea47e3916fde6ec36.intermediate
Traceback (most recent call last):
  File "tools/js2c.py", line 234, in <module>
    main()
  File "tools/js2c.py", line 219, in main
    js_files = utils.SearchFiles(options.directory, 'js')
  File "/home/gireesh/node/tools/utils.py", line 112, in SearchFiles
    list = glob.glob(dir+ '/**/*.' + ext, recursive=True)
TypeError: glob() got an unexpected keyword argument 'recursive'
  LINK(target) /home/gireesh/node/out/Release/gen-regexp-special-case
make: *** [libnode.target.mk:13: /home/gireesh/node/out/Release/obj/gen/node_javascript.cc] Error 1

@targos
Copy link
Member

targos commented Jul 20, 2021

This should have been fixed by #39293

@gireeshpunathil
Copy link
Member Author

This should have been fixed by #39293

@targos - unfortunately, does not look like it fixed it, I am still getting the same error.

@legendecas
Copy link
Member

legendecas commented Jul 20, 2021

I was wondering if the ./configure used a different version (as printed above as 3.8.10) with the build step of tools/js2c.py (probably 2.x or 3.5 lower).

@legendecas
Copy link
Member

legendecas commented Jul 20, 2021

Build step of node_js2c: https://github.com/nodejs/node/blob/master/node.gyp#L820
Used in tools/js2c.py: https://github.com/nodejs/node/blob/master/tools/js2c.py#L219

So the problem here is that #39293 is hoisting the result of search files. However, node_js2c did not use the variable but the directories instead for shorter argument lines. While js2c.py is invoked with default or whatever python that version is lower than 3.5 then it works in an unexpected way.

@gireeshpunathil
Copy link
Member Author

@legendecas - I applied this patch (#39465) and could resolve the issue, thanks!

legendecas added a commit that referenced this issue Jul 23, 2021
To help core contributors identify which file is missing from the build.

PR-URL: #39460
Refs: #39408
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this issue Jul 25, 2021
To help core contributors identify which file is missing from the build.

PR-URL: #39460
Refs: #39408
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@legendecas
Copy link
Member

@gireeshpunathil As the comment #39465 (comment) suggests, can you check out the content of ./config.mk (which is generated by ./configure), and if out/tool/bin/python presents and where does it linking to? I tested on a Linux system with both python2 and python3 installed and python in PATH is linking to python2, sadly the problem didn't present.

@gireeshpunathil
Copy link
Member Author

cat config.mk

# Do not edit. Generated by the configure script.
BUILDTYPE=Release
NODE_TARGET_TYPE=executable
PYTHON=/usr/bin/python3.9
PREFIX=/usr/local
[gireesh@u1 node]$ ls -lrt out/tool/bin/python
ls: cannot access 'out/tool/bin/python': No such file or directory
[gireesh@u1 node]$ which python
/usr/bin/python
[gireesh@u1 node]$ which python3
/usr/bin/python3
[gireesh@u1 node]$ ls -lrt `which python`
lrwxrwxrwx 1 root root 16 Jul 16 06:11 /usr/bin/python -> /usr/bin/python3
[gireesh@u1 node]$ ls -lrt `which python3`
lrwxrwxrwx 1 root root 25 Jul 16 05:36 /usr/bin/python3 -> /etc/alternatives/python3
[gireesh@u1 node]$ ls -lrt /etc/alternatives/python3
lrwxrwxrwx 1 root root 18 Jul 16 05:36 /etc/alternatives/python3 -> /usr/bin/python3.9
[gireesh@u1 node]$ ls -lrt /usr/bin/python3.9
-rwxr-xr-x 1 root root 7728 Mar 10 09:32 /usr/bin/python3.9
[gireesh@u1 node]$ file /usr/bin/python3.9
/usr/bin/python3.9: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=0ba4590c6f56cbe0efce22e0d4224a0f527489f9, stripped
[gireesh@u1 node]$ 

@legendecas - here is the dump of config.mk as well as the details of various python installations. Though it (the soft link business) looks little funny to me, I don't see any evident issue with the whole setup, but will leave it to you to comment!

@legendecas
Copy link
Member

@gireeshpunathil it seems that python2 is not even in your path, so ./configure doesn't generate bin overrides in config.mk and out/tool/bin/python. I'm wondering if there is any python2 installation in your local environment and the build script somehow to reveal that? 🧐

@gireeshpunathil
Copy link
Member Author

@legendecas - no, I don't have python2 in my path, as I never installed one. Am I supposed to? The build instructions at https://github.com/nodejs/node/blob/master/BUILDING.md#note-about-python do not specify to do so?

@legendecas
Copy link
Member

@gireeshpunathil No, you don't have to install python2. I'm just wondering where did the python2 pop out during the build and how do we override the python in PATH.

richardlau pushed a commit that referenced this issue Jul 29, 2021
To help core contributors identify which file is missing from the build.

PR-URL: #39460
Refs: #39408
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
BethGriggs pushed a commit that referenced this issue Jul 29, 2021
To help core contributors identify which file is missing from the build.

PR-URL: #39460
Refs: #39408
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this issue Aug 2, 2021
PR-URL: #39465
Fixes: #39408
Fixes: #39456
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos pushed a commit that referenced this issue Aug 2, 2021
PR-URL: #39465
Fixes: #39408
Fixes: #39456
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos pushed a commit that referenced this issue Sep 4, 2021
To help core contributors identify which file is missing from the build.

PR-URL: #39460
Refs: #39408
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this issue Sep 4, 2021
PR-URL: #39465
Fixes: #39408
Fixes: #39456
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants