Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault crash when using prisma client when using PostgreSQL #10649

Closed
SCDerox opened this issue Dec 11, 2021 · 113 comments · Fixed by #18426
Closed

Segmentation fault crash when using prisma client when using PostgreSQL #10649

SCDerox opened this issue Dec 11, 2021 · 113 comments · Fixed by #18426

Comments

@SCDerox
Copy link

SCDerox commented Dec 11, 2021

Bug description

When trying to use findMany() (or any other method) on a model, prisma client crashes and the following line appears in my log:
Segmentation fault (core dumped).

This only happens if I use PostgreSQL as far as I know, this first appeared when I switched to PostgreSQL from mysql.

How to reproduce

  1. Initiate a new prisma project (make sure to use PostgreSQL) and generate the client - no errors there
  2. Write a simple client which fetches some data from the databse
  3. The script crashes and Segmentation fault (core dumped) gets logged in the console

Expected behavior

The script does not crash.

Prisma information

Example schema:

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

model User {
    id String @id
    email String?
}

Example test client:

const {PrismaClient} = require('@prisma/client');
const prisma = new PrismaClient({ log: ['query', 'info', 'warn'] })
prisma.user.findMany().then((e) => {
    console.log(e)
})

Environment & setup

  • OS: Pop!_OS (Linux pop-os 5.15.5-76051505-generic #202111250933~1638201579~21.04~09f1aa7-Ubuntu SMP Tue Nov 30 02: x86_64 x86_64 x86_64 GNU/Linux)
  • Database: PostgreSQL v14.1 (Ubuntu 14.1-1.pgdg21.04+1)
  • Node.js version: v17.2.0

Prisma Version

prisma                  : 3.6.0
@prisma/client          : 3.6.0
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine dc520b92b1ebb2d28dc3161f9f82e875bd35d727 (at node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
Migration Engine        : migration-engine-cli dc520b92b1ebb2d28dc3161f9f82e875bd35d727 (at node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Introspection Engine    : introspection-core dc520b92b1ebb2d28dc3161f9f82e875bd35d727 (at node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x)
Format Binary           : prisma-fmt dc520b92b1ebb2d28dc3161f9f82e875bd35d727 (at node_modules/@prisma/engines/prisma-fmt-debian-openssl-1.1.x)
Default Engines Hash    : dc520b92b1ebb2d28dc3161f9f82e875bd35d727
Studio                  : 0.440.0
@SCDerox SCDerox added the kind/bug A reported bug. label Dec 11, 2021
@sveisvei
Copy link

Happens here as well, pre- 3.6.0 does not have this issue. M1 mac

@janpio janpio added topic: query-crash team/client Issue for team Client. bug/0-unknown Bug is new, does not have information for reproduction or reproduction could not be confirmed. labels Dec 17, 2021
@janpio
Copy link
Member

janpio commented Dec 17, 2021

Any idea how to reproduce? If this was a universal things, we would have hundred of reports instead of just 2 of course - so something needs to trigger that. @sveisvei Can you please also post all the versions of all the things you are using?

@SCDerox
Copy link
Author

SCDerox commented Dec 26, 2021

I still don't know what the issue could be, but I have followed this guide to debug this issue a little bit more. I have to admit, I don't have any experience with this, but I have attached crash.log file which got generated by the segfault-handler module:

PID 19523 received SIGSEGV for address: 0x1000001b7
/home/simonc/workspace/prisma-test/node_modules/segfault-handler/build/Release/segfault-handler.node(+0x372d)[0x7f92370cf72d]
/lib/x86_64-linux-gnu/libc.so.6(+0x46520)[0x7f9236bc4520]
/lib/x86_64-linux-gnu/libssl.so.1.1(SSL_get_peer_certificate+0x17)[0x7f92342e4d87]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x1410427)[0x7f921565d427]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xf37b0d)[0x7f9215184b0d]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x10d8010)[0x7f9215325010]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xf7551e)[0x7f92151c251e]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xff5e62)[0x7f9215242e62]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xffdd3d)[0x7f921524ad3d]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xfd66cd)[0x7f92152236cd]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc4755d)[0x7f9214e9455d]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc6ce8b)[0x7f9214eb9e8b]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc7cd46)[0x7f9214ec9d46]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc0f64a)[0x7f9214e5c64a]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc18023)[0x7f9214e65023]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc5b5d8)[0x7f9214ea85d8]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc43c2a)[0x7f9214e90c2a]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc65e87)[0x7f9214eb2e87]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc629a4)[0x7f9214eaf9a4]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0xc526a7)[0x7f9214e9f6a7]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x20a71a)[0x7f921445771a]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x20916b)[0x7f921445616b]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x1b1529)[0x7f92143fe529]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13d23d6)[0x7f921561f3d6]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13d18e8)[0x7f921561e8e8]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13bfc65)[0x7f921560cc65]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13d0d2c)[0x7f921561dd2c]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13d669f)[0x7f921562369f]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13cac68)[0x7f9215617c68]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13c9926)[0x7f9215616926]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13be02a)[0x7f921560b02a]
/home/simonc/workspace/prisma-test/node_modules/.prisma/client/libquery_engine-debian-openssl-1.1.x.so.node(+0x13d5f78)[0x7f9215622f78]

Maybe this helps

@SCDerox
Copy link
Author

SCDerox commented Dec 27, 2021

Okay, well, I spend another (at least) four hours playing with things and got it to work: I basically only had to build my own engines - it works now ^^
Maybe this could help @sveisvei too 😅

@janpio
Copy link
Member

janpio commented Dec 27, 2021

So you analysis would be that the libquery_engine-debian-openssl-1.1.x.so.node that is used by default is not compatible with the OS you are using?

@SCDerox
Copy link
Author

SCDerox commented Dec 27, 2021

Yes, probably

@tisuela
Copy link

tisuela commented Jan 2, 2022

Yes, probably

I am also getting a segmentation fault when using .findmany, except only when I use supabase for the PostgreSQL server.

My segfault crash logs also point to the same engine issue. I'm on Ubuntu 20.04. Can I get more details on the engine you are now using? @SCDerox

@tisuela
Copy link

tisuela commented Jan 2, 2022

I just fixed this by using the binary engine instead of the library engine. Will there ever be a proper fix for this?

@janpio
Copy link
Member

janpio commented Jan 2, 2022

If you can provide a proper reproduction, we could potentially follow what is happening and try to fix it. Without a reproduction, it will be impossible. As @SCDerox 's initial problem probably was related to their operating system, and yours is clearly not, please open a new issue and provide all the information the bug report template asks for @tisuela.

@sveisvei
Copy link

sveisvei commented Jan 3, 2022

Tested with a fresh install of prisma-client @ 3.7.0, and the error did not occur anymore.

@janpio
Copy link
Member

janpio commented Jan 3, 2022

Let's close this then 👍

@sveisvei If Prisma not working on Pop!_OS is a general thing, could you maybe create an issue with reproduction steps so we can confirm this is the case and then potentially fix this? Thanks.

@janpio janpio closed this as completed Jan 3, 2022
@huzaifah-esm
Copy link

Tested with a fresh install of prisma-client @ 3.7.0, and the error did not occur anymore.

I'm still getting this error after doing this.

I just fixed this by using the binary engine instead of the library engine. Will there ever be a proper fix for this?

@tisuela How do I do this?

Environment: Ubuntu 20.04 LTS running on Windows WSL2

Logs:

> studio
> prisma studio -b none --schema schema.prisma

Environment variables loaded from .env
Prisma schema loaded from schema.prisma
Prisma Studio is up on http://localhost:5555
Segmentation fault

@janpio
Copy link
Member

janpio commented Jan 9, 2022

Can you share your project? Is this reproducible on other computer?

@huzaifahj
Copy link

huzaifahj commented Jan 9, 2022

@janpio Here are the steps to reproduce this. I used a fresh VM (Ubuntu 20.04 LTS) on Google Compute Engine. I used a fresh PostgreSQL database (try Supabase for a quick test).

npm init -y
npm install -D prisma
npm install @prisma/client
nano schema.prisma

schema.prisma:

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = "URL"
}

model User {
  id             String       @id @default(uuid())
  createdAt      DateTime     @default(now())
  updatedAt      DateTime     @updatedAt
}
npx prisma generate
npx prisma studio

Now try and view a model on localhost:5555...

Prisma Studio frontend logs:

Prisma Client Error
Unable to run script
Message: Error in HTTP Request (Status: 502)
Query:
[object Object]

Console logs:

Prisma schema loaded from schema.prisma
Prisma Studio is up on http://localhost:5555
Segmentation fault (core dumped)

Error also seen with Ubuntu 18.04 LTS on Windows WSL2. I cannot reproduce this on my home setup (Arch Linux).

Error could not be fixed by changing engine to binary (cc @tisuela)

generator client {
  provider = "prisma-client-js"
  engineType = "binary"
}

@tisuela
Copy link

tisuela commented Jan 9, 2022

Tested with a fresh install of prisma-client @ 3.7.0, and the error did not occur anymore.

I'm still getting this error after doing this.

I just fixed this by using the binary engine instead of the library engine. Will there ever be a proper fix for this?

@tisuela How do I do this?

Environment: Ubuntu 20.04 LTS running on Windows WSL2

Logs:

> studio
> prisma studio -b none --schema schema.prisma

Environment variables loaded from .env
Prisma schema loaded from schema.prisma
Prisma Studio is up on http://localhost:5555
Segmentation fault

@janpio Here are the steps to reproduce this. I used a fresh VM (Ubuntu 20.04 LTS) on Google Compute Engine. I used a fresh PostgreSQL database (try Supabase for a quick test).

npm init -y
npm install -D prisma
npm install @prisma/client
nano schema.prisma

schema.prisma:

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = "URL"
}

model User {
  id             String       @id @default(uuid())
  createdAt      DateTime     @default(now())
  updatedAt      DateTime     @updatedAt
}
npx prisma generate
npx prisma studio

Now try and view a model on localhost:5555...

Prisma Studio frontend logs:

Prisma Client Error
Unable to run script
Message: Error in HTTP Request (Status: 502)
Query:
[object Object]

Console logs:

Prisma schema loaded from schema.prisma
Prisma Studio is up on http://localhost:5555
Segmentation fault (core dumped)

Error also seen with Ubuntu 18.04 LTS on Windows WSL2. I cannot reproduce this on my home setup (Arch Linux).

Error could not be fixed by changing engine to binary (cc @tisuela)

generator client {
  provider = "prisma-client-js"
  engineType = "binary"
}

I also rolled back to prisma 3.5.x

@janpio janpio reopened this Jan 10, 2022
@huzaifahj
Copy link

Not sure if this helps but I tried using Google Cloud SQL for PostgreSQL but having the same issues:

[218]: [1-1] db=postgres,user=postgres LOG:  could not receive data from client: Connection reset by peer

Error persists when using version 3.5.0 for prisma CLI and client:

Environment variables loaded from .env                                                                                  
Prisma schema loaded from schema.prisma                                                                                 
Prisma Studio is up on http://localhost:5555
┌─────────────────────────────────────────────────────────┐                                                             
│  Update available 3.5.0 -> 3.7.0                        
│                                                             
│  Run the following to update                            
│                                                             
│    npm i --save-dev prisma@latest                       
│                                                             
│    npm i @prisma/client@latest                          
│                                                             
└─────────────────────────────────────────────────────────┘                                                             
Segmentation fault

@janpio
Copy link
Member

janpio commented Jan 10, 2022

Note that the later comments might actually describe a different problem, as your error message does not include the (core dumped) string that was originally mentioned to be part of the `Segmentation fault" error message. But I guess it is to late now to split into its own, separate issue.

@huzaifahj
Copy link

So far the messages on WSL2 have been Segmentation fault, and the messages on a Ubuntu VM have been Segmentation fault (core dumped)

@pantharshit00
Copy link
Contributor

I just tried this on WSL 2 and I wasn't able to reproduce this there:
image

I will spin up a PopOS VM now to try this there.

@pantharshit00
Copy link
Contributor

image

I am also unable to reproduce this in PopOS. Can you maybe try with LTS version of node here?

@huzaifahj
Copy link

huzaifahj commented Jan 12, 2022

@pantharshit00 Ah, the issue is fixed for me (on Ubuntu 20.04 WSL2) after updating to Node 17 LTS. Previously I was on Node 14 LTS.

If you can replicate this on the lower Node version it may be worth refreshing the docs:

Note: You need Node.js (version 12.6 or higher) for this tutorial.
https://github.com/prisma/docs/blob/main/content/100-getting-started/01-quickstart.mdx

@janpio
Copy link
Member

janpio commented Jan 12, 2022

If this is failing on older Node versions, we can hopefully reproduce with that and then figure out what is going on.

@louistiti
Copy link

Hi, I'm having the same issue. Here is my prisma -v output:

prisma                  : 4.12.0
@prisma/client          : 4.12.0
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine 659ef412370fa3b41cd7bf6e94587c1dfb7f67e7 (at node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
Migration Engine        : migration-engine-cli 659ef412370fa3b41cd7bf6e94587c1dfb7f67e7 (at node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Format Wasm             : @prisma/prisma-fmt-wasm 4.12.0-67.659ef412370fa3b41cd7bf6e94587c1dfb7f67e7
Default Engines Hash    : 659ef412370fa3b41cd7bf6e94587c1dfb7f67e7
Studio                  : 0.483.0

I'm running it on an Ubuntu 22.04.2 LTS.

aqrln added a commit that referenced this issue Apr 14, 2023
@janpio janpio added this to the 4.13.0 milestone Apr 15, 2023
@sarink
Copy link

sarink commented Apr 17, 2023

Experiencing seg faults running Ubuntu server 22 on a raspberry pi 4 and using the latest prisma (4.12). Downgrading to prisma 4.0 fixed it for me.

@aqrln
Copy link
Member

aqrln commented Apr 17, 2023

@sarink the new ARM problem is a different issue from this one, could you please leave a comment in #18510?

@aqrln
Copy link
Member

aqrln commented Apr 17, 2023

(just a heads up that the link was wrong in the previous comment in case you are reading it in email notifications and not in GitHub UI, updated it #18510)

@mrsufgi
Copy link

mrsufgi commented Apr 17, 2023

For those having issues with segmentation fault (I got them on upsert calls) , after upgrading to 4.8.0+ with alpine.
remove the openssl1.1-compat from your container. this resolved this issue for me.

My migrations container:

FROM --platform=linux/amd64 docker.io/node:19-alpine3.17

WORKDIR /app

RUN apk add --update --no-cache libc6-compat 

COPY prisma ./prisma
COPY package*.json ./
COPY .yarn ./.yarn
COPY yarn.lock .yarnrc.yml ./
RUN yarn set version canary
RUN yarn

CMD yarn prisma migrate deploy; yarn prisma db seed

@louistiti
Copy link

Hi, I'm having the same issue. Here is my prisma -v output:

prisma                  : 4.12.0
@prisma/client          : 4.12.0
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine 659ef412370fa3b41cd7bf6e94587c1dfb7f67e7 (at node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
Migration Engine        : migration-engine-cli 659ef412370fa3b41cd7bf6e94587c1dfb7f67e7 (at node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Format Wasm             : @prisma/prisma-fmt-wasm 4.12.0-67.659ef412370fa3b41cd7bf6e94587c1dfb7f67e7
Default Engines Hash    : 659ef412370fa3b41cd7bf6e94587c1dfb7f67e7
Studio                  : 0.483.0

I'm running it on an Ubuntu 22.04.2 LTS.

Installing the following version temporarily solved it for me:

npm install --save-dev prisma@4.12.0-integration-rtld-deepbind.3
npm install --save @prisma/client@4.12.0-integration-rtld-deepbind.3

@aqrln
Copy link
Member

aqrln commented Apr 17, 2023

This issue is fixed in Prisma 4.13.0 which will be released tomorrow, on Tuesday, Apr 18. If you need to temporarily use a pre-release version until then, you can use 4.13.0-dev.33.

Note that this GitHub issue is about a specific bug that occurs when using Node.js 17+ on a system with OpenSSL 1.1 on x86-64 CPUs when using TLS to connect to database. Please do not post comments here if you encounter a segfault under different circumstances, and open a new issue instead!

If you are facing segfaults on ARM, please leave a comment in #18510 or open a new issue.

@janpio
Copy link
Member

janpio commented Apr 18, 2023

@mrsufgi Can you please open a new issue for the configuration where something went wrong for you, which you could fix by removing openssl1.1-compat from your container? That should not be happening and we want to take a deeper look. Thanks.

@aqrln
Copy link
Member

aqrln commented Apr 18, 2023

@janpio that actually sounds expected: if openssl1.1-compat was present in the container, we'd detect the platform as musl and use OpenSSL 1.1, so this issue would occur. Without openssl1.1-compat we'd detect the platform as musl-openssl-3.0.x and use OpenSSL 3.0, so the issue wouldn't occur as the OpenSSL we'd use would be ABI compatible with one provided by Node.js. So no need to open a new issue since it's exactly the same issue.

@kylejw2
Copy link

kylejw2 commented Jun 19, 2023

To anyone else using linux-musl as their platform, upgrading prisma versions wasn't enough for me. I also had to add engineType = "binary" to my client in the schema.prisma file

@Hampfh
Copy link

Hampfh commented Jul 29, 2023

I've spent many hours trying to figure out why Prisma is segfaulting in the query engine, have tried changing the engine format to "engine" but without any promising result unfortunately.

Prisma 5.0.0
Platform Dockerized through MacOS M1 (ARM)
NodeJS 18.17.0

I tried playing around with different versions and found that node 18.16 and above has this problem, from what I can tell node version 18.15 is the latest version that runs without encountering any segfault. Worth noting is that the faulty versions did still work for smaller queries but they broke when GraphQL concatenated larger queries which I had a test case for.

So basically this was the workaround I came up with:

# Dockerfile
FROM node:18 -> FROM node:18.15

@Jolg42
Copy link
Member

Jolg42 commented Aug 1, 2023

@Hampfh Could you create a new issue for this? This issue is now closed since a few months, and we do not re-open a closed issue when a new similar problem comes up, as it might be unrelated to the original issue.
So with a new "fresh" issue, we can then try a reproduction and investigate.

@aqrln
Copy link
Member

aqrln commented Aug 1, 2023

@Hampfh this is very valuable context for #18510, could you please leave this comment there? This issue is about a different problem that only occurred on x86_64, and is now fixed.

@kazbeel
Copy link

kazbeel commented Aug 4, 2023

I just fixed this by using the binary engine instead of the library engine. Will there ever be a proper fix for this?

This error still happens with version 4.15.0. (NodeJS v18.11.0) engineType = "binary" does the trick.

@rokkoo
Copy link

rokkoo commented Oct 15, 2023

We spend almost full day trying to find why was not working until We find this post, We was building our image using node:18.4.0-alpine.

At the end we could fix the problem by decreading the node version to node:18.alpine thanks! @kylejw2 and @Hampfh

@razzeee
Copy link

razzeee commented Oct 18, 2023

I was able to fix this for us by going from node:18-alpine3.16 to node:18-alpine3.18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment