tsc --watch fails with JavaScript heap out of memory error

[samchon]

Bug description

Related issue: #10183 (comment) (@aqrln)

When run tsc --watch command and edit some codes using prisma, about two minutes being consumed, and such error being occurred. For reference, such bug does not occur when running only tsc command. Also, does not occur in tsc --watch command until do update, either.

• tsc: works fine
• tsc --watch and do not anything: works fine
• tsc --watch and update something: error

I also tried to avoid the bug by assigning much more memory, but same bug still occurs, and needed much more minutes to get the same error message.

• node --max-old-space-size=4096 node_modules/typescript/bin/tsc --watch

[9:05:36 PM] File change detected. Starting incremental compilation...


<--- Last few GCs --->

[6216:000001C592CE9A30]   135933 ms: Mark-Compact 4045.6 (4133.3) -> 4030.8 (4134.3) MB, 808.22 / 0.00 ms  (average mu = 0.107, current mu = 0.031) allocation failure; scavenge might not succeed
[6216:000001C592CE9A30]   136760 ms: Mark-Compact 4046.6 (4134.3) -> 4031.9 (4135.5) MB, 798.19 / 0.00 ms  (average mu = 0.072, current mu = 0.035) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF723D45B3F node_api_throw_syntax_error+202639
 2: 00007FF723CC2366 v8::base::CPU::num_virtual_address_bits+66502
 3: 00007FF723CC4181 v8::base::CPU::num_virtual_address_bits+74209
 4: 00007FF72472B4B1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF724714C18 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF7245765B0 v8::Platform::SystemClockTimeMillis+659552
 7: 00007FF724582833 v8::Platform::SystemClockTimeMillis+709347
 8: 00007FF724580194 v8::Platform::SystemClockTimeMillis+699460
 9: 00007FF7245732D0 v8::Platform::SystemClockTimeMillis+646528
10: 00007FF72458894A v8::Platform::SystemClockTimeMillis+734202
11: 00007FF7245891C7 v8::Platform::SystemClockTimeMillis+736375
12: 00007FF724597B43 v8::Platform::SystemClockTimeMillis+796147
13: 00007FF7242581D5 v8::CodeEvent::GetFunctionName+116821
14: 00007FF6C479AAFA

How to reproduce

I don't know how to reproduce it.

Such bug had not occurred when project is small, but started being happened when project being larger.

As this project is commercial project that is not opened public, I'm sorry to cannot share with you.

If a reproducible repo must be, I'll try it next month \o/.

Expected behavior

No problem when running tsc --watch and updating source code.

Prisma information

Sorry for cannot write prisma schema content due to commercial project.

Environment & setup

• OS: Windows 11
• Database: Postgres v15
• Node.js version: v20.4.0

Prisma Version

prisma                  : 5.0.0
@prisma/client          : 5.0.0
Current platform        : windows
Query Engine (Node-API) : libquery-engine 6b0aef69b7cdfc787f822ecd7cdc76d5f1991584 (at node_modules\@prisma\engines\query_engine-windows.dll.node)
Schema Engine           : schema-engine-cli 6b0aef69b7cdfc787f822ecd7cdc76d5f1991584 (at node_modules\@prisma\engines\schema-engine-windows.exe)
Schema Wasm             : @prisma/prisma-schema-wasm 4.17.0-26.6b0aef69b7cdfc787f822ecd7cdc76d5f1991584
Default Engines Hash    : 6b0aef69b7cdfc787f822ecd7cdc76d5f1991584
Studio                  : 0.487.0

[GitMarijn]

Same

[justinsmid]

👍 - I have also noticed the same error. For us, it started happening after we upgraded prisma 4.15.0 -> 5.0.0.
For us however, it also happens during a regular tsc, not just with tsc --watch.
Our prisma.schema is over 6000 lines long, so that probably has something to do with it.
In our case, NODE_OPTIONS=--max-old-space-size=5120 does seem to help prevent the error when running tsc, but with that band-aid fix it's only a matter of time before we run into it again.

[samchon]

@justinsmid My commercial project also has the 6K LOC prisma schema definition (TS is 100K).

But in my case, I cannot avoid the bug even when assigning 16GB memory. Therefore, I also downgraded to 4 version.

Is there someone who can provide reproducible repo?

[liqMats]

Same behaviour while using prisma client extensions. TypeScript v5.1.6

[millsp]

Hey @samchon @liqMats @justinsmid @GitMarijn thanks for the report. Could you please provide us access to a small reproduction or directly to you repository. We can sign an NDA if needed for you to keep things private, and/or you can contact me via DM on our public Slack or Discord.

Also, which of you is using client extensions, and who is not?

[phil-loops]

This may be a long-shot, but do you have the incremental key defined in your tsconfig? Ran into a potentially similar issue and resolved by explicitly setting it to false.

We were also using the single pattern in our prisma declaration file -- commenting out all the info relating to global also seemed to help.

/*  
 * this is the file where I declare the global singleton for prisma
 * this is part of next.js (13) project
 */

import { PrismaClient } from "@prisma/client";

// declare global {
//   // eslint-disable-next-line no-var
//   var prisma: PrismaClient | undefined;
// }

const prisma = global.prisma || new PrismaClient();

if (process.env.NODE_ENV !== "production") global.prisma = prisma;

export default prisma;

[millsp]

@phil-loops are you using client extensions?

[phil-loops]

@millsp no, in this repo we were initializing prisma with new PrismaClient(). FWIW, in my case, my npx tsc command is also failing from the jump when I have a tsconfig.buildinfo artifact file.

For me, the tsc --watch command appears to fail consistently when changes are made to the a file which directly (or indirectly) imports the @prisma/client.

Also, I noted when I deleted node_modules/@prisma/client folder, the compiler would be angry (300ish errors), but it wouldn't hit FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

[phil-loops]

Steps to reproduce:

1. delete the existing tsconfig.buildinfo artifact file (if present)
2. In a branch on prisma 5, run {command}
   1. add export const noop = null; to a file that depends on @prisma/client
   2. run {command} one more time

command --> rm -f -r node_modules/@prisma; npm i; npx prisma generate; sleep 5; npx tsc

explanation:

• clear the downloaded prisma files
• install dependencies
• emit the current typescript definitions
• take a chill (vaguely aware of the existence of the concept of the typescript language server -- think it may take some time for it to fully catch up to all changes. Dunno.)
• compile the app

For me, the last time running the command (during the compiling step), I hit the fatal memory error.

Fwiw we also have a sizable prisma file (we have 40 models and 5 enums defined in our schema). For now, I think we're going to delete the artifact file and set incremental to false in our tsconfig.

[millsp]

Thanks for the info. We also have some huge schemas at hand, so I could try with that and get back to you. If I cannot reproduce, I'll need your help again :)

[millsp]

Does 4.16 work well for you?

[phil-loops]

Ah, nice point. 4.16.0 seems to be okay. However, 4.16.1 seems to be where the behavior was introduced.

[revmischa]

Upgrading from 4.12.0 to 5.0.0 and I've started getting tsc OOM errors with both tsc --build --incremental and with --watch. Takes a very long time (2m) before this error pops out.
Not using client extension but I do have some middleware.

❯ time px tsc

<--- Last few GCs --->

[3244107:0x6dbd530]    64683 ms: Mark-sweep 4042.2 (4132.5) -> 4027.1 (4133.3) MB, 833.3 / 0.0 ms  (average mu = 0.100, current mu = 0.019) allocation failure; scavenge might not succeed
[3244107:0x6dbd530]    65451 ms: Mark-sweep 4042.9 (4133.3) -> 4027.8 (4134.0) MB, 757.9 / 0.0 ms  (average mu = 0.060, current mu = 0.012) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb6e500 node::Abort() [node]
 2: 0xa7e632  [node]
 3: 0xd47ea0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xd48247 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xf25605  [node]
 6: 0xf26508 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xf36a13  [node]
 8: 0xf37888 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xf121ee v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xf135b7 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
11: 0xef478a v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
12: 0x12b7d2f v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
13: 0x16e9979  [node]
 ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL  Command was killed with SIGABRT (Aborted): tsc
pnpm exec tsc  118.07s user 2.75s system 180% cpu 1:06.89 total

[millsp]

Thanks for the report @revmischa. Could you please give me access to the repo so that I can reproduce this?

Update: We've received a reproduction.

[millsp]

Since I had two reports via DM that this is fixed on our @prisma/client@dev tag (= main branch), I am closing this issue. We have received reports of enums being impacted by recent changes (unrelated to the fix for this issue) and also have a fix up for review. If you want to try the "type oom" fix + "enum" fix, you can install this version 5.1.0-integration-fix-client-enums.1.

Thanks all of you.

[peterarusanoff]

I am on Prisma version 5.3 and client 5.3, and I am Still getting this error

this is happening on my Mac with 16 GB RAM
our schema file is only 600 lines long and filled with many enums as well

[millsp]

Hey @peterarusanoff 👋 I'd appreciate if you could create a new issue for us with the necessary info for us to reproduce it. Thanks!