Failed to start the query engine: Connection refused #74

AndryHTC · 2022-12-07T14:11:18Z

PrismaClientInitializationError:
  message: Failed to start the query engine: Connection refused
  errorCode: null
  clientVersion: 2.4.5

#0      BinaryEngine._createProcess (package:orm/src/engine_core/binary/binary_engine_io.dart:441:7)
<asynchronous suspension>
#1      BinaryEngine.start (package:orm/src/engine_core/binary/binary_engine_io.dart:333:17)
<asynchronous suspension>
#2      BinaryEngine.request (package:orm/src/engine_core/binary/binary_engine_io.dart:290:5)
<asynchronous suspension>
#3      ModelDelegate.findMany (package:package_name/prisma_client.dart:55755:42)
<asynchronous suspension>
...

The database is PlanetScale, so I don't have any Docker compose file to set up with SSL or other...
Running the Dart Server locally I have no problems, but in staging/production gives me that error (the connection strings are the same).

@medz Anything to suggest?

Originally posted by @AndryHTC in #55 (comment)

The text was updated successfully, but these errors were encountered:

medz · 2022-12-08T06:57:22Z

@AndryHTC This needs to be investigated why it is rejected, and my usual practice is to manually start the Prisma query engine in the command line window to locate the problem.

AndryHTC · 2022-12-08T12:16:36Z

@yahu1031

AndryHTC · 2022-12-09T09:52:59Z

@AndryHTC This needs to be investigated why it is rejected, and my usual practice is to manually start the Prisma query engine in the command line window to locate the problem.

I will not refuse a little bit of a guide on how to do it 😄

syedzainqadri · 2022-12-09T21:55:42Z

Hay Facing the same issue
with mysql in development
PrismaClientInitializationError:
message: Failed to start the query engine: Connection refused
errorCode: null
clientVersion: 2.4.5
Can you guide that how can we manually start the prisma query engine

medz · 2022-12-12T02:17:32Z

The Prisma query engine is placed in the .dart_tool/prisma directory by default

AndryHTC · 2022-12-12T11:04:54Z

I'm sorry @medz, but you aren't suggesting anything useful... Like @EwigSol sad

Can you guide that how can we manually start the prisma query engine

We understand where is the engine... The question is: how can we try to start it? Is a binary without any extension...

Please, give us support 🙏

medz · 2022-12-12T19:05:08Z

@AndryHTC @EwigSol Sorry, I understand. Below I will fully describe the specific debugging methods and steps of "Connection refused" in Prisma ORM in the deployed environment

OpenSSL/LibSSL

In the deployment environment, check whether OpenSSL or LibSSL has been installed, and if not, install it.

unix socket blocking

If your database is running locally, then you need to check whether it is blocked by the local unix socket.

Usually manifested as: Connection refused or Connection timed out.

Remote database connection

If your database is running remotely, then you need to check if it is a database connection problem.

Usually you can try to ping or use telnet to check if you can connect to the database.

Local database

If your database is running locally, then you need to check whether it is a problem with the database connection.

Prisma binary engine default location

The default location of the Prisma binary engine is .dart_tool/prisma/query-engine. On Windows, the default location is .dart_tool\prisma\query-engine.exe.

Start parameters

--debug: start debug mode, you can see more debug information
--enable-raw-queries: enable raw query mode
--enable-open-telemetry: enable OpenTelemetry mode
--enable-playground: enable GraphQL Playground
--datamodel: Prisma schema.prisma content Base64 encoded string, choose one of --datamodel-path
--datamodel-path: Prisma schema.prisma file path, choose one of --datamodel
--overwrite-datasources: Overwrite database sources
--port: start port

The startup parameters in Prisma ORM for Dart are:

PRISMA_DML={Base64 encoded prisma schema content} query-engine --enable-raw-queries --enable-metrics --enable-open-telemetry --port {free port}

Manually start the binary engine

DATABASE_URL={DB connect url} query-engine --enable-raw-queries --enable-playground --enable-metrics --enable-open-telemetry --port {port} --datamodel-path {path to schema.prisma}

medz · 2022-12-12T19:09:46Z

Meanwhile, I'm working on a more stable C api query engine, using Rust. However, my personal time is limited, and I can only maintain the binary query engine version for the time being. I think I need help from the community. I have developed a version, but it has not been applied to Prisma ORM for Dart, because it depends on the Prisma engines warehouse (using git for dependency, non-standard release) which leads to huge maintenance costs. The query engine requirement of C API is submitted to Prisma official. Use FFI for data interaction to achieve a more stable engine and Flutter support.

https://github.com/odroe/prisma-query-c-api

AndryHTC · 2022-12-15T11:01:32Z

The default location of the Prisma binary engine is .dart_tool/prisma/query-engine. On Windows, the default location is .dart_tool\prisma\query-engine.exe.

First of all... On Windows, I don't find any query-engine.exe, just query-engine exactly like all the other OSes...

The startup parameters in Prisma ORM for Dart are:

PRISMA_DML={Base64 encoded prisma schema content} query-engine --enable-raw-queries --enable-metrics --enable-open-telemetry --port {free port}

Manually start the binary engine

DATABASE_URL={DB connect url} query-engine --enable-raw-queries --enable-playground --enable-metrics --enable-open-telemetry --port {port} --datamodel-path {path to schema.prisma}

I've tried to manually start the binary engine in local - with Ubuntu, MacOS and Windows with WSL - where Prisma works. I've also tried to manually start the binary engine in the deployed environment (where Prisma do NOT work).

All the commands says that the query-engine command is not found...

Local MacOS (where Prisma Works):

project_server % cd .dart_tool 
.dart_tool % cd prisma
prisma % ls                                                                                                                                                                                                                                                                
query-engine

prisma % DATABASE_URL=[url...] query-engine --enable-raw-queries --enable-playground --enable-metrics --enable-open-telemetry --port [port...] --datamodel-path ../../prisma/schema.prisma
zsh: command not found: query-engine

Local Ubuntu (where Prisma Works):

$ cd project_server/
project_server$ cd .dart_tool/
project_server/.dart_tool$ cd prisma
project_server/.dart_tool/prisma$ ls
query-engine

project_server/.dart_tool/prisma$ DATABASE_URL=[url...] query-engine --enable-raw-queries --enable-playground --enable-metrics --enable-open-telemetry --port [port...] --datamodel-path ../../prisma/schema.prisma
query-engine: command not found

Amazon Linux 2 (where Prisma do NOT work):

[project_server]$ cd prisma

[project_server/prisma]$ ls
development.dart  prisma_version.dart  query-engine  schema.prisma

[project_server/prisma]$ DATABASE_URL=[url...] query-engine --enable-raw-queries --enable-playground --enable-metrics --enable-open-telemetry --port [port...] --datamodel-path schema.prisma
bash: query-engine: command not found
[project_server/prisma]$ query-engine
bash: query-engine: command not found

I've lost an entire day trying to start the query-engine even where it works... I don't understand if I'm doing something wrong or if you missed something in the reply.

By the way thank you for replying @medz

medz · 2022-12-15T14:02:03Z

@AndryHTC I know what you missed, the download of the query engine is a progressive download based on the demand of the cli command. So if you want to manually debug the binary engine, please use the precache command to download the binary engine. For details, please see https://prisma.pub/reference/cli#precache

medz · 2022-12-15T14:10:20Z

Replace executable binary name in command line：

DATABASE_URL=[url...] ./query-engine\
  --enable-raw-queries\
  --enable-playground --enable-metrics \
  --enable-open-telemetry --port [port...] \
  --datamodel-path ../../prisma/schema.prisma

AndryHTC · 2022-12-15T22:32:54Z

@medz thank you

Replace executable binary name in command line：

At lest we can go further 😄
./query-engine: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

medz · 2022-12-16T00:41:08Z

error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

@AndryHTC It means that the running environment lacks OpenSSL/LibSSL. My suggestion is to download the query engine in the running environment (CLI will search for the matching version of the local OpenSSL/LibSSL). If this prompt appears after the successful download, it means that OpenSSL/LibSSL is found locally but not sure Its version was then rolled back to LibSSL 1.0 (which is the minimum requirement for the Prisma binary engine to run)

Repair suggestion:

Run openssl version -v to check whether there is this command or a version higher than 1.0 locally
If step 1 is unsuccessful, run ls -l /lib64 && ls -l /usr/lib64 to see what matches it in the output: libssl.so.(\d+.\d+.\d+)
Try to install OpenSSL
After completing step 3, delete the locally cached query engine binary, and use the dart run orm precache command to re-download the binary engine

AndryHTC · 2022-12-17T09:28:31Z

Run openssl version -v to check whether there is this command or a version higher than 1.0 locally

[ec2-user@ip-**]$ openssl version -v
OpenSSL 1.0.2k-fips  26 Jan 2017

AndryHTC · 2022-12-17T09:47:50Z

@medz I've tried to re-run the dart run orm precache -t query as you shown in other issues, but:

[ec2-user@ip-** serverpod_server]$ dart run orm precache -t query
Dart_LoadScriptFromKernel: The binary program does not contain 'main'.

Is this talking about the project folder? It is a serverpod_server one, so there is no main.dart. The start script is the lib/server.dart

medz · 2022-12-17T12:22:43Z

AndryHTC · 2022-12-17T13:21:09Z

... so? How this could help me in any way? 🤣

I will attach the same if you want

sh-4.2$ sudo su ec2-user
[ec2-user@ip-** bin]$ cd
[ec2-user@ip-** ~]$ cd serverpod
[ec2-user@ip-**1 serverpod]$ cd active
[ec2-user@ip-** active]$ cd serverpod_server
[ec2-user@ip-** serverpod_server]$ ls
analysis_options.yaml  aws  bin  CHANGELOG.md  config  docker-compose.yaml  generated  lib  prisma  pubspec.lock  pubspec.yaml  README.md  setup-tables.cmd  web
[ec2-user@ip-** serverpod_server]$ dart run orm precache -t query
Dart_LoadScriptFromKernel: The binary program does not contain 'main'.
[ec2-user@ip-** serverpod_server]$ cd prisma
[ec2-user@ip-** prisma]$ ls
development.dart  schema.prisma

AndryHTC · 2022-12-17T13:26:39Z

@medz cleaning dart cache e applying dart pub get the command starts but freezes the server instance (the console and all the calls to the server too)

[ec2-user@ip-** serverpod_server]$ dart pub get
Resolving dependencies... (2.5s)
  _fe_analyzer_shared 50.0.0 (51.0.0 available)
  analyzer 5.2.0 (5.3.1 available)
  build_runner 2.3.2 (2.3.3 available)
  googleapis 7.0.0 (9.2.0 available)
  intl 0.17.0 (0.18.0 available)
  js 0.6.5 (0.6.6 available)
  matcher 0.12.13 (0.12.14 available)
  path 1.8.2 (1.8.3 available)
  serverpod 0.9.20 (0.9.21 available)
  serverpod_auth_server 0.9.20 (0.9.21 available)
  serverpod_client 0.9.20 (0.9.21 available)
  serverpod_serialization 0.9.20 (0.9.21 available)
  serverpod_service_client 0.9.20 (0.9.21 available)
  serverpod_shared 0.9.20 (0.9.21 available)
Got dependencies!
[ec2-user@ip-** serverpod_server]$ dart run orm precache -t query
Building package executable... (15.4s)

AndryHTC · 2022-12-18T16:52:58Z

@medz I apologize for being so insistent, but the project is in production and I need to know if I have to fallback into something else

medz · 2022-12-20T05:55:52Z

@AndryHTC I'm sorry for not being able to reply to you these few days, because I was infected with COVID-19 these days and I just got better today. I will carefully analyze the entire issue context and mentioned content later, and then create a new example repository to reproduce this bug together.

medz · 2022-12-20T06:14:38Z

@AndryHTC I created a new warehouse to deal with this issue, and I have invited you to enter the warehouse (invitation link: https://github.com/odroe/prisma-dart-issue-74/invitations) to submit what you encountered Is there minimal code to reproduce the problem? Then I'd watch your code looking for problems.

medz · 2022-12-20T06:15:59Z

The best way is to write a simple deployment dockerfile, so that it is easier for me to grasp the running target environment information to collect problems.

AndryHTC · 2022-12-20T07:16:04Z

@AndryHTC I'm sorry for not being able to reply to you these few days, because I was infected with COVID-19 these days and I just got better today. I will carefully analyze the entire issue context and mentioned content later, and then create a new example repository to reproduce this bug together.

@medz you don't have to justify yourself. This is open source software you are not obligated to reply my questions 😊

So... I extremely appreciate your help, and I hope you feel better now.

Can we communicate just for this issue through an IM service? LinkedIn it's ok to me

AndryHTC · 2022-12-20T09:15:36Z

@AndryHTC I created a new warehouse to deal with this issue, and I have invited you to enter the warehouse (invitation link: https://github.com/odroe/prisma-dart-issue-74/invitations) to submit what you encountered Is there minimal code to reproduce the problem? Then I'd watch your code looking for problems.

I'm having some difficulty getting my minimal code published. It seems that I need to package a good chunk of the project to make it work, which can be a bit overwhelming. I'm using the Serverpod framework and have set up the infrastructure with a Terraform script on AWS. The deployment is triggered by a GitHub action that is chained to the Serverpod and AWS infrastructure secrets...

And at the end, I'm running into an issue only with the final output of these actions.

Do you have any suggestions or ideas that might help me out to provide you the minimal code to reproduce the issue?

medz · 2022-12-20T09:41:54Z

@AndryHTC I created a new warehouse to deal with this issue, and I have invited you to enter the warehouse (invitation link: https://github.com/odroe/prisma-dart-issue-74/invitations) to submit what you encountered Is there minimal code to reproduce the problem? Then I'd watch your code looking for problems.

I'm having some difficulty getting my minimal code published. It seems that I need to package a good chunk of the project to make it work, which can be a bit overwhelming. I'm using the Serverpod framework and have set up the infrastructure with a Terraform script on AWS. The deployment is triggered by a GitHub action that is chained to the Serverpod and AWS infrastructure secrets...

And at the end, I'm running into an issue only with the final output of these actions.

Do you have any suggestions or ideas that might help me out to provide you the minimal code to reproduce the issue?

My only suggestion is to use Docker to write a deployment architecture similar to your actual environment to implement reproduction. Because any runtime environment architecture can be simulated by writing a dockerfile.

This approach requires you to be familiar with Docker and to be able to write dockerize proficiently and successfully.

medz · 2022-12-21T14:38:46Z

prisma/prisma#16871 Perhaps my code downloaded the wrong binary engine on the target platform, I will update the code to find the SSL version and release a new version to verify this conjecture.

medz · 2022-12-21T14:43:42Z

prisma/prisma#16553 (comment)

AndryHTC · 2022-12-21T20:01:01Z

prisma/prisma#16553 (comment)

This could be the key to everything... I'll give it a try! 👍
@medz I appreciate all you efforts

#74

medz · 2022-12-22T07:41:44Z

@AndryHTC Hey, I released 2.4.7-dev.0 refactored platform detection and published it as a separate package.

Can you try to upgrade to version 2.4.7-dev.0 to test whether it is fixed?

AndryHTC · 2022-12-26T11:04:12Z

Can you try to upgrade to version 2.4.7-dev.0 to test whether it is fixed?

No joy...

prisma/prisma#16553 (comment)

This could be the key to everything... I'll give it a try! 👍 @medz I appreciate all you efforts

I cannot upgrade my Amazon-Linux-2's openssl version to v.3.0.7

medz · 2022-12-26T13:15:39Z

Can you try to upgrade to version 2.4.7-dev.0 to test whether it is fixed?

No joy...

prisma/prisma#16553 (comment)

This could be the key to everything... I'll give it a try! 👍 @medz I appreciate all you efforts

I cannot upgrade my Amazon-Linux-2's openssl version to v.3.0.7

It's such a pity and I feel bad for it.

I'll keep trying to find what's triggering this issue.

AndryHTC · 2022-12-27T11:27:51Z

I was able to get it running in the current instance, even with version 2.4.6. I'm not sure how I managed to do it. I'm worried that if the EC2 instance is terminated and re-instantiated, the problem will occur again.

syedzainqadri · 2023-02-12T08:09:24Z

How can i do this on a docker container. Should i just pass all of this in my CMD or As a run command i mean this does not make any clear sence to me

syedzainqadri · 2023-02-12T14:01:44Z

I was able to get it running in the current instance, even with version 2.4.6. I'm not sure how I managed to do it. I'm worried that if the EC2 instance is terminated and re-instantiated, the problem will occur again.

can you refrence me through this as i am not able to run the prisma engine manually and it is not working by default. on my docker container.

medz · 2023-03-01T01:48:59Z

@AndryHTC @syedzainqadri @aschulz90 @jacobaraujo7 https://github.com/odroe/prisma-dart/blob/main/example/simple/Dockerfile I have solved this problem! This dockerfile is for your reference.

AndryHTC · 2023-04-25T18:53:54Z

@AndryHTC @syedzainqadri @aschulz90 @jacobaraujo7 https://github.com/odroe/prisma-dart/blob/main/example/simple/Dockerfile I have solved this problem! This dockerfile is for your reference.

@medz Here we are again 😢. We've managed to not touch the production code in the last 2 months but after an Instance destroy/creation here the problem is back.

The Dockerfile is not available anymore... Can you provide the info? How this applies to orm 3.3.0? What is the right time to generate the engine and the client when deploying?

medz · 2023-04-26T02:02:14Z

@AndryHTC https://github.com/odroe/prisma-dart/blob/main/example/Dockerfile I wrote a new

medz · 2023-04-26T02:04:20Z

@AndryHTC https://github.com/odroe/prisma-dart/blob/main/Dockerfile Here is a dockerfile that complements the official Dart landscape. Used to satisfy Prisma binary engine runtime dependencies in scratch.

AndryHTC · 2023-04-28T10:20:12Z

@AndryHTC https://github.com/odroe/prisma-dart/blob/main/Dockerfile Here is a dockerfile that complements the official Dart landscape. Used to satisfy Prisma binary engine runtime dependencies in scratch.

Thank you. I would like to know what are the minimum steps needed to get it working and the explanation of each one 🙏🏼

medz added a commit that referenced this issue Dec 22, 2022

chore(get-platform): Add package

93f9255

#74

medz mentioned this issue Feb 12, 2023

Deploying prisma made dart app to dockers #117

Closed

medz closed this as completed Mar 1, 2023

Failed to start the query engine: Connection refused #74

Failed to start the query engine: Connection refused #74

Comments

AndryHTC commented Dec 7, 2022 • edited

medz commented Dec 8, 2022

AndryHTC commented Dec 8, 2022

AndryHTC commented Dec 9, 2022

syedzainqadri commented Dec 9, 2022 • edited

medz commented Dec 12, 2022

AndryHTC commented Dec 12, 2022 • edited

medz commented Dec 12, 2022 • edited

OpenSSL/LibSSL

unix socket blocking

Remote database connection

Local database

Prisma binary engine default location

Start parameters

Manually start the binary engine

medz commented Dec 12, 2022

AndryHTC commented Dec 15, 2022

Manually start the binary engine

medz commented Dec 15, 2022

medz commented Dec 15, 2022 • edited

AndryHTC commented Dec 15, 2022

medz commented Dec 16, 2022 • edited

AndryHTC commented Dec 17, 2022

AndryHTC commented Dec 17, 2022

medz commented Dec 17, 2022

AndryHTC commented Dec 17, 2022

AndryHTC commented Dec 17, 2022

AndryHTC commented Dec 18, 2022

medz commented Dec 20, 2022

medz commented Dec 20, 2022

medz commented Dec 20, 2022

AndryHTC commented Dec 20, 2022 • edited

AndryHTC commented Dec 20, 2022

medz commented Dec 20, 2022

medz commented Dec 21, 2022

medz commented Dec 21, 2022

AndryHTC commented Dec 21, 2022

medz commented Dec 22, 2022

AndryHTC commented Dec 26, 2022

medz commented Dec 26, 2022

AndryHTC commented Dec 27, 2022

syedzainqadri commented Feb 12, 2023

syedzainqadri commented Feb 12, 2023

medz commented Mar 1, 2023

AndryHTC commented Apr 25, 2023 • edited

medz commented Apr 26, 2023

medz commented Apr 26, 2023

AndryHTC commented Apr 28, 2023

AndryHTC commented Dec 7, 2022 •

edited

syedzainqadri commented Dec 9, 2022 •

edited

AndryHTC commented Dec 12, 2022 •

edited

medz commented Dec 12, 2022 •

edited

medz commented Dec 15, 2022 •

edited

medz commented Dec 16, 2022 •

edited

AndryHTC commented Dec 20, 2022 •

edited

AndryHTC commented Apr 25, 2023 •

edited