Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pnpm list output is truncated from v.8.9.0 #7378

Open
2 of 4 tasks
missvalentinep opened this issue Dec 5, 2023 · 3 comments · May be fixed by #7431
Open
2 of 4 tasks

pnpm list output is truncated from v.8.9.0 #7378

missvalentinep opened this issue Dec 5, 2023 · 3 comments · May be fixed by #7431
Labels
type: bug

Comments

@missvalentinep
Copy link

missvalentinep commented Dec 5, 2023 •
edited

Verify latest release

  • I verified that the issue exists in the latest pnpm release

pnpm version

>= 8.9.0

Which area(s) of pnpm are affected? (leave empty if unsure)

CLI

Link to the code that reproduces this issue or a replay of the bug

No response

Reproduction steps

Run pnpm ls "@company-scope/**" --depth Infinity --parseable

Describe the Bug

The output of this command changes from v8.8.0 to v8.9.0. In 8.9.0 some packages are now missing.

I believe it is related to this MR #7122

It's supposed to prune the tree only for pnpm why, but pnpm list was also affected.
It happens because when we call pnpm ls "@company-scope/**", listForPackages (patched in this commit method) is being called.

If I execute pnpm ls --depth Infinity --parseable the missing packages are there.

Expected Behavior

Output is not truncated

Which Node.js version are you using?

20.10.0

Which operating systems have you used?

  • macOS
  • Windows
  • Linux

If your OS is a Linux based, which one it is? (Include the version if relevant)

No response
@btea btea linked a pull request Dec 16, 2023 that will close this issue
Open
@zkochan
Copy link
Member

zkochan commented Dec 21, 2023

It's supposed to prune the tree only for pnpm why, but pnpm list was also affected.

These are the same under the hood. pnpm why pkg is the same as running pnpm ls pkg --depth Infinity. We had to add some limitations to avoid out of memory errors.

cc @nachoaldamav

@missvalentinep
Copy link
Author

We had to add some limitations to avoid out of memory errors.

But is it possible to revert (or to add a way to somehow avoid) these limitations for pnpm list? We had some important functionality that depended on the output of this command, and as a workaround we had to pin the pnpm version for now

@gluxon gluxon mentioned this issue Jan 24, 2024
Open
4 tasks
@busywhistling
Copy link

Hi, for pnpm v8.14.3 (on macOS m2 Ventura 13.6.4 (22G513)), running pnpm ls --json --long --depth=Infinity either crashes with/without nvm [1,2] or outputs an incorrect string [3]. I suppose this is related to this issue here. As such, due to this I'm blocked from integrating it with the cyclonedx-pnpm package to parse pnpm dependency tree to generate SBOM/vulnerability info...

[1] pnpm crashes in one of my repos while using nvm 
<--- Last few GCs --->

[4575:0x120008000]    14700 ms: Mark-Compact 4006.4 (4137.8) -> 4002.2 (4137.8) MB, 1490.83 / 0.00 ms  (average mu = 0.432, current mu = 0.050) allocation failure; scavenge might not succeed
[4575:0x120008000]    17187 ms: Mark-Compact 4017.9 (4137.8) -> 4014.6 (4165.3) MB, 2474.96 / 0.00 ms  (average mu = 0.190, current mu = 0.005) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1041ecbf4 node::Abort() [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 2: 0x1041ecddc node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 3: 0x104370da8 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 4: 0x1045456e8 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 5: 0x1045441c4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 6: 0x10453a9dc v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 7: 0x10453b23c v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 8: 0x104520240 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
 9: 0x104907e70 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
10: 0x104c64c44 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
11: 0x104c13038 Builtins_AsyncFunctionAwaitUncaught [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
12: 0x109c74ffc 
13: 0x109c5fb94 
14: 0x104c13210 Builtins_AsyncFunctionAwaitResolveClosure [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
15: 0x104cc0fb8 Builtins_PromiseFulfillReactionJob [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
16: 0x104c02b94 Builtins_RunMicrotasks [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
17: 0x104bda3f4 Builtins_JSRunMicrotasksEntry [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
18: 0x1044b29f0 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
19: 0x1044b2edc v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
20: 0x1044b30b8 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
21: 0x1044da284 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
22: 0x1044daa20 v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
23: 0x10411cc64 node::InternalCallbackScope::Close() [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
24: 0x10411d01c node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
25: 0x10413348c node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
26: 0x1041f264c node::fs::FSReqCallback::Resolve(v8::Local<v8::Value>) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
27: 0x1041f2d9c node::fs::AfterNoArgs(uv_fs_s*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
28: 0x1041e7cf0 node::MakeLibuvRequestCallback<uv_fs_s, void (*)(uv_fs_s*)>::Wrapper(uv_fs_s*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
29: 0x104bb8b64 uv__work_done [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
30: 0x104bbc5b4 uv__async_io [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
31: 0x104bce68c uv__io_poll [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
32: 0x104bbcb78 uv_run [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
33: 0x10411d754 node::SpinEventLoopInternal(node::Environment*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
34: 0x10422c8d8 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
35: 0x10422c674 node::NodeMainInstance::Run() [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
36: 0x1041b7030 node::Start(int, char**) [/Users/paramjit/.nvm/versions/node/v20.10.0/bin/node]
37: 0x18538ff28 start [/usr/lib/dyld]
fish: Job 1, 'pnpm ls --json --long --depth=I…' terminated by signal SIGABRT (Abort)
[2] pnpm crashes in one of my repos without nvm 
<--- Last few GCs --->

[4690:0x148008000]    12559 ms: Scavenge 4007.0 (4116.3) -> 4006.7 (4126.8) MB, 13.50 / 0.00 ms  (average mu = 0.830, current mu = 0.779) allocation failure; 
[4690:0x148008000]    12577 ms: Scavenge 4013.5 (4126.8) -> 4013.0 (4127.5) MB, 14.50 / 0.00 ms  (average mu = 0.830, current mu = 0.779) allocation failure; 
[4690:0x148008000]    12776 ms: Scavenge 4014.4 (4127.5) -> 4013.6 (4149.0) MB, 198.75 / 0.00 ms  (average mu = 0.830, current mu = 0.779) allocation failure; 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

1: 0x10317dd8c node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
2: 0x1032f9bd4 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
3: 0x1032f9b84 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
4: 0x10349f924 v8::internal::Heap::CallGCPrologueCallbacks(v8::GCType, v8::GCCallbackFlags, v8::internal::GCTracer::Scope::ScopeId) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
5: 0x10349e830 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
6: 0x103496c7c v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
7: 0x103497420 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
8: 0x10347f420 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
9: 0x10377b534 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
10: 0x102f8f954 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/opt/homebrew/Cellar/node/21.6.1/bin/node]
11: 0x10b0f6ed4 
12: 0x10b0f74b8 
13: 0x102f04b84 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/21.6.1/bin/node]
14: 0x102f04b84 Builtins_InterpreterEntryTrampoline [/opt/homebrew/Cellar/node/21.6.1/bin/node]
15: 0x102f3bb10 Builtins_AsyncFunctionAwaitResolveClosure [/opt/homebrew/Cellar/node/21.6.1/bin/node]
16: 0x102ff52d8 Builtins_PromiseFulfillReactionJob [/opt/homebrew/Cellar/node/21.6.1/bin/node]
17: 0x102f2a654 Builtins_RunMicrotasks [/opt/homebrew/Cellar/node/21.6.1/bin/node]
18: 0x102f02794 Builtins_JSRunMicrotasksEntry [/opt/homebrew/Cellar/node/21.6.1/bin/node]
19: 0x103419d44 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
20: 0x10341a508 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
21: 0x10343bbe0 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
22: 0x10343ba18 v8::internal::MicrotaskQueue::PerformCheckpointInternal(v8::Isolate*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
23: 0x10309cb90 node::InternalCallbackScope::Close() [/opt/homebrew/Cellar/node/21.6.1/bin/node]
24: 0x10309d0e8 node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
25: 0x1030b4814 node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
26: 0x10318422c node::fs::FSReqCallback::Resolve(v8::Local<v8::Value>) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
27: 0x103185ae4 node::fs::AfterNoArgs(uv_fs_s*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
28: 0x1031759d8 node::MakeLibuvRequestCallback<uv_fs_s, void (*)(uv_fs_s*)>::Wrapper(uv_fs_s*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
29: 0x1063871b8 uv__work_done [/opt/homebrew/Cellar/libuv/1.47.0/lib/libuv.1.dylib]
30: 0x10638a800 uv__async_io [/opt/homebrew/Cellar/libuv/1.47.0/lib/libuv.1.dylib]
31: 0x10639a1e8 uv__io_poll [/opt/homebrew/Cellar/libuv/1.47.0/lib/libuv.1.dylib]
32: 0x10638ac94 uv_run [/opt/homebrew/Cellar/libuv/1.47.0/lib/libuv.1.dylib]
33: 0x10309d96c node::SpinEventLoopInternal(node::Environment*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
34: 0x1031c52a8 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
35: 0x1031c5010 node::NodeMainInstance::Run() [/opt/homebrew/Cellar/node/21.6.1/bin/node]
36: 0x10313dfac node::Start(int, char**) [/opt/homebrew/Cellar/node/21.6.1/bin/node]
37: 0x18538ff28 start [/usr/lib/dyld]
fish: Job 1, 'pnpm ls --json --long --depth=I…' terminated by signal SIGABRT (Abort)
[3] pnpm outputs invalid string in another repo 
{
  "error": {
    "code": "pnpm",
    "message": "Invalid string length"
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(reviewing): pnpm list output should not be truncated btea/pnpm
3 participants
@zkochan @missvalentinep @busywhistling