bug: docs - Docker node:16-alpine requires setting engineType = "binary" for Prisma to work

[nicolomaioli]

Provide environment information

System:
OS: Linux 6.0 Fedora Linux 37 (Workstation Edition)
CPU: (16) x64 AMD Ryzen 7 PRO 6850U with Radeon Graphics
Memory: 23.97 GB / 30.12 GB
Container: Yes
Shell: 5.9 - /bin/zsh
Binaries:
Node: 16.19.0 - ~/.volta/tools/image/node/16.19.0/bin/node
npm: 8.19.3 - ~/.volta/tools/image/node/16.19.0/bin/npm

ct3aMetadata.initVersion: 6.11.3

Describe the bug

I've added a Dockerfile (copy-pasted from the docs, using node:16-alpine as a base image) and docker-compose.yml to a freshly bootstrapped app. I modified index.tsx to run trpc.example.getAll.useQuery(), and received this error message at runtime:

Listening on port 3000 url: http://localhost:3000
prisma:error
Invalid `prisma.example.findMany()` invocation:


Unable to load Node-API Library from /app/node_modules/.prisma/client/libquery_engine-linux-musl.so.node, Library may be corrupt

• I got a similar error with node:16-buster-slim (pointing to the Debian engine).
• Changing the base image to node:16-buster fixes the error.
• Adding engineType = "binary" to Prisma generator client also fixes the error under node:16-alpine.

To reproduce

• Bootstrap application with trpc and prisma:

npm create t3-app@6.11.3

• Follow steps 1-3 from the docs
• Add trpc.example.getAll.useQuery() to index.tsx
• Use the following minimal docker-compose.yml:

version: "3.9"
services:
  app:
    platform: "linux/amd64"
    build:
      context: .
      dockerfile: Dockerfile
      args:
        NEXT_PUBLIC_CLIENTVAR: "clientvar"
    working_dir: /app
    ports:
      - "3000:3000"
    image: t3-app
    environment:
      - DATABASE_URL=mysql://foo:password@db:3306/bar
  db:
    image: mysql:8
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    ports:
      - 3306:3306
    environment:
      MYSQL_USER: foo
      MYSQL_PASSWORD: password
      MYSQL_DATABASE: bar
      MYSQL_RANDOM_ROOT_PASSWORD: true
    volumes:
      - db:/var/lib/mysql

volumes:
  db: {}

• Run docker compose up -d --build
• Visit localhost:3000, then check the logs

Additional information

It's worth noting that setting engineType = "binary" is not a recommended approach according to the Prisma docs as it introduces additional overhead in the architecture: Query Engine. If there's a problem with using the Node-API Library in Alpine that seems out of scope for the t3 docs, but might be worth mentioning.

[c-ehrlich]

Seems like it might be related to this? prisma/prisma#16205

Seems like it can be "fixed" by using node:16-alpine3.16 or node:18-alpine3.16 instead of the 3.17 versions. Could you confirm whether or not that works, without having to apply the engineType = "binary" setting?

If so I would recommend putting this in the docs as a temporary solution, and then keeping an eye on it as hopefully there will be a better solution soon.

[nicolomaioli]

Hi @c-ehrlich, I think you're right! Setting node:16-alpine3.16 with openssl instead of openssl1.1-compat and no enginetype = "binary" also seems to work. Let me know if there's anything else I can do to help!

[josethz00]

@nicolomaioli hi, how can I set openssl over openssl1.1-compat ? I have tried node:16-alpine3.16 without success

[nicolomaioli]

@nicolomaioli hi, how can I set openssl over openssl1.1-compat ? I have tried node:16-alpine3.16 without success

Change this line in the Dockerfile:

- RUN apk add --no-cache libc6-compat openssl1.1-compat
+ RUN apk add --no-cache libc6-compat openssl

Edit: I believe Alpine 3.17 uses v3 of openssl, which is why you need to specify 1.1. The -compat I think has to do with the fact that Alpine uses musl rather than glibc. Hopefully this helps!

[josethz00]

Change this line in the Dockerfile:

- RUN apk add --no-cache libc6-compat openssl1.1-compat
+ RUN apk add --no-cache libc6-compat openssl

Edit: I believe Alpine 3.17 uses v3 of openssl, which is why you need to specify 1.1. The -compat I think has to do with the fact that Alpine uses musl rather than glibc. Hopefully this helps!

Thank you @nicolomaioli , it works! Using node:16-alpine3.16 and openssl

[nicolomaioli]

@c-ehrlich I'd be happy to raise a PR for the documentation if that's helpful, I haven't contributed before but it seems straightforward enough

[c-ehrlich]

@c-ehrlich I'd be happy to raise a PR for the documentation if that's helpful, I haven't contributed before but it seems straightforward enough

First time contributions are always welcome, especially after opening such a thorough issue :)

Just so I understand correctly:

• 3.16 with openssl works
• 3.17 with openssl-compat and binary works
• other combinations don't work (in particular 3.17 with openssl-compat but not setting binary)
  is this correct?

If so I'm not sure what the best thing is to recommend. engineType = "binary" seems bad if Prisma recommends to avoid it. But remaining locked to 3.16 seems irresponsible as people will be opting out of security updates this way. In the long run we should probably ditch Alpine if there is no other option.

What do you (and everyone else) think is the best short-mid term solution here?

[nicolomaioli]

Thanks @c-ehrlich happy to take a shot at this.

Debian Buster also works without setting engineType to binary, so that might be the simplest fix and most straightforward recommendation. I need to play around with Buster slim as that would be a better replacement for Alpine.

We could add a paragraph for Alpine, and explain the trade-offs (size, performance, security), I think there’s enough to unpack there. I’m a little wary about polluting the docs though, so open to suggestions. It’s worth mentioning that Alpine 3.16 won’t reach EOL until mid-2024, so it still receives security patches. Also I would argue that the real issue there is Prisma’s lack of support for OpenSSL past v1.1.

Finally, we should consider how to best track the open issue on Prisma. Not sure what the best approach is here, maybe leaving an open issue to track?

[c-ehrlich]

Thanks @c-ehrlich happy to take a shot at this.

Debian Buster also works without setting engineType to binary, so that might be the simplest fix and most straightforward recommendation. I need to play around with Buster slim as that would be a better replacement for Alpine.

We could add a paragraph for Alpine, and explain the trade-offs (size, performance, security), I think there’s enough to unpack there. I’m a little wary about polluting the docs though, so open to suggestions. It’s worth mentioning that Alpine 3.16 won’t reach EOL until mid-2024, so it still receives security patches. Also I would argue that the real issue there is Prisma’s lack of support for OpenSSL past v1.1.

Finally, we should consider how to best track the open issue on Prisma. Not sure what the best approach is here, maybe leaving an open issue to track?

Sorry for the delayed reply. I think either of the following is fine:

1. Pin Alpine to 3.16
2. use engineType = "binary" (not for everyone, just as a step in the Docker instructions)
   Regardless of which we choose, return to this in 6 months - hopefully Prisma has sorted it by then.

I'm leaning towards 1, but I'm far from an expert on Docker stuff so feel free to open a PR for whichever you think is the better choice :)

[nicolomaioli]

Thanks @c-ehrlich happy to take a shot at this.
Debian Buster also works without setting engineType to binary, so that might be the simplest fix and most straightforward recommendation. I need to play around with Buster slim as that would be a better replacement for Alpine.
We could add a paragraph for Alpine, and explain the trade-offs (size, performance, security), I think there’s enough to unpack there. I’m a little wary about polluting the docs though, so open to suggestions. It’s worth mentioning that Alpine 3.16 won’t reach EOL until mid-2024, so it still receives security patches. Also I would argue that the real issue there is Prisma’s lack of support for OpenSSL past v1.1.
Finally, we should consider how to best track the open issue on Prisma. Not sure what the best approach is here, maybe leaving an open issue to track?

Sorry for the delayed reply. I think either of the following is fine:

1. Pin Alpine to 3.16
2. use engineType = "binary" (not for everyone, just as a step in the Docker instructions)
   Regardless of which we choose, return to this in 6 months - hopefully Prisma has sorted it by then.

I'm leaning towards 1, but I'm far from an expert on Docker stuff so feel free to open a PR for whichever you think is the better choice :)

No worries at all, I've also been a little swamped with the holidays!

I agree that Alpine 3.16 is the more reasonable suggestion, especially if the plan is to revise in 6 months. I'll get the PR going!

[jkomyno]

Hi, I'm one of the Prisma maintainers. I second what @nicolomaioli has stated here.
Please read this comment in a Prisma issue if you're interested in why this issue happened and how to truly solve it

[nicolomaioli]

Hi, I'm one of the Prisma maintainers. I second what @nicolomaioli has stated here.

Please read this comment in a Prisma issue if you're interested in why this issue happened and how to truly solve it

Thanks! When I first tried to get this working with 3.17 I had the listed dependency and still couldn't make it work. I wonder now if the fact that I was using MySQL could have had an impact.

I'm happy to do some more digging over the weekend, would it be ok for me to reach out with more findings/questions?

[jkomyno]

@nicolomaioli Certo! Feel free to reach out again opening a new Prisma issue, although I think I know why it didn't work initially with alpine:3.17. Prisma depends on some Rust engines which are compiled for specific operating systems and/or openssl versions. As you can see in our docs, we only started supporting openssl-3.0.x for Linux Alpine starting from prisma@4.8.0, which we released right before Christmas 2022. You have probably opened this issue just a few days before that

[nicolomaioli]

@nicolomaioli Certo! Feel free to reach out again opening a new Prisma issue, although I think I know why it didn't work initially with alpine:3.17. Prisma depends on some Rust engines which are compiled for specific operating systems and/or openssl versions. As you can see in our docs, we only started supporting openssl-3.0.x for Linux Alpine starting from prisma@4.8.0, which we released right before Christmas 2022. You have probably opened this issue just a few days before that

That's brilliant, thank you! I'll try the different engines should be pretty straightforward. And thanks for reaching out! 🙏

[c-ehrlich]

@jkomyno thanks for coming in here and clarifying :)

@nicolomaioli we are already on Prisma 4.8.0 now so this should hopefully be an easy fix now. If it works for you, would you be up for opening a pr for it?

[nicolomaioli]

@jkomyno thanks for coming in here and clarifying :)

@nicolomaioli we are already on Prisma 4.8.0 now so this should hopefully be an easy fix now. If it works for you, would you be up for opening a pr for it?

Happy to 🙌

[nicolomaioli]

@jkomyno @c-ehrlich I just managed to run on Alpine 3.17, with Prisma v4.8.0, using the MySQL driver. Note that my AMD laptop is currently out for repair (😢), so I'm running on a MacBook Air M1 using platform linux/amd64. This is the Dockerfile:

##### DEPENDENCIES

FROM --platform=linux/amd64 node:16-alpine3.17 AS deps
WORKDIR /app

# Install Prisma Client - remove if not using Prisma

COPY prisma ./

# Install dependencies based on the preferred package manager

COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml\* ./

RUN \
	if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
	elif [ -f package-lock.json ]; then npm ci; \
	elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm i; \
	else echo "Lockfile not found." && exit 1; \
	fi

##### BUILDER

FROM --platform=linux/amd64 node:16-alpine3.17 AS builder
ARG DATABASE_URL
ARG NEXT_PUBLIC_CLIENTVAR
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

# ENV NEXT_TELEMETRY_DISABLED 1

RUN \
	if [ -f yarn.lock ]; then SKIP_ENV_VALIDATION=1 yarn build; \
	elif [ -f package-lock.json ]; then SKIP_ENV_VALIDATION=1 npm run build; \
	elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && SKIP_ENV_VALIDATION=1 pnpm run build; \
	else echo "Lockfile not found." && exit 1; \
	fi

##### RUNNER

FROM --platform=linux/amd64 node:16-alpine3.17 AS runner
WORKDIR /app

ENV NODE_ENV production

# ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/next.config.mjs ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json

COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs
EXPOSE 3000
ENV PORT 3000

CMD ["node", "server.js"]

docker-compose.yml:

services:
  web:
    depends_on:
      - db
    build:
      context: .
      dockerfile: Dockerfile
    working_dir: /app
    ports:
      - "8080:8080"
    image: t3mysql
    environment:
      - PORT=8080
      - DATABASE_URL=mysql://test:password@db:3306/test
  db:
    image: mysql:8
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    ports:
      - 3306:3306
    environment:
      MYSQL_DATABASE: test
      MYSQL_USER: test
      MYSQL_PASSWORD: password
      MYSQL_RANDOM_ROOT_PASSWORD: true
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
      - db:/var/lib/mysql
volumes:
  db: {}

Note the absence of additional dependencies: no libc6-compat, and no openssl1.1-compat as the engines now support Openssl 3x and musl. Crucially, this only works in AMD64 as documented here.

I am keen to test this with PostgreSQL and SQLite also, I know very little about MongoDB tho so I could use some pointers to test this quickly.

[c-ehrlich]

@jkomyno @c-ehrlich I just managed to run on Alpine 3.17, with Prisma v4.8.0, using the MySQL driver. Note that my AMD laptop is currently out for repair (😢), so I'm running on a MacBook Air M1 using platform linux/amd64. This is the Dockerfile:

This seems great! I just got it running locally (M1 Pro) and can confirm it works. I benchmarked the Docker x64 emulation a while ago and the overhead was something like 5-10% depending on the task. Given how powerful the M1 is I think this is ok for now, and we don't exactly have a good alternative anyway.

Re: alpine version: I think it might be better to not pin to 3.17? What do you think?

I'm afraid I don't know much about Mongo either as I've never used it outside of toy projects, but

1. I would expect that adapting this setup to other databases is no different than it would be with any other Next.js/Prisma app
2. The ratio of SQL to Mongo users of CT3A is probably something like 100:1, so I think if there is some other challenge to it, it's fine to have this become another issue once someone runs into it

A bigger issue to me is that the existing Docker setup feels like it is specifically for a production setup - multi stage build that only includes the output, not running migrations when the containers come up, etc. However people who deploy dockerized apps in my experience usually also develop in a dockerized environment, so it would be nice to enable that.

I had a working dev Docker setup a few months ago (ie optimized for being able to see changes quickly, making schema changes quickly, etc), but it was a completely separate Dockerfile and docker-compose, which afaik isn't really considered best practices anymore (?). It was extremely simple and could be improved upon but I could see if I still have the repo/branch as a starting point.

[nicolomaioli]

This seems great! I just got it running locally (M1 Pro) and can confirm it works. I benchmarked the Docker x64 emulation a while ago and the overhead was something like 5-10% depending on the task. Given how powerful the M1 is I think this is ok for now, and we don't exactly have a good alternative anyway.

Re: alpine version: I think it might be better to not pin to 3.17? What do you think?

That's great! Really good to hear it worked for you as well.

I think having a separate Dockerfile and docker-compose is still standard, at least in my experience with Kubernetes.

Re Alpine 3.17, I think the advantage there is that it comes with OpenSSL 3x instead of OpenSSL 1.1:

3.16

3.17

@jkomyno please correct me if I'm wrong there, be also good to get your opinion here.

[c-ehrlich]

This seems great! I just got it running locally (M1 Pro) and can confirm it works. I benchmarked the Docker x64 emulation a while ago and the overhead was something like 5-10% depending on the task. Given how powerful the M1 is I think this is ok for now, and we don't exactly have a good alternative anyway.

Re: alpine version: I think it might be better to not pin to 3.17? What do you think?

That's great! Really good to hear it worked for you as well.

I think having a separate Dockerfile and docker-compose is still standard, at least in my experience with Kubernetes.

Re Alpine 3.17, I think the advantage there is that it comes with OpenSSL 3x instead of OpenSSL 1.1:

3.16

3.17

@jkomyno please correct me if I'm wrong there, be also good to get your opinion here.

I definitely agree with 3.17. My question is whether we want to pin to 3.17 or not pin at all, ie automatically use 3.18 once that is released (seems to be a ~6 month version schedule). My general opinion is that if a new version breaks something I'd rather know this sooner than later, but I can also see an argument for pinning to a known working version.

Also sorry for editing your post, I clicked the wrong thing and thought I was quoting it 😅 should be back to normal now

[nicolomaioli]

This seems great! I just got it running locally (M1 Pro) and can confirm it works. I benchmarked the Docker x64 emulation a while ago and the overhead was something like 5-10% depending on the task. Given how powerful the M1 is I think this is ok for now, and we don't exactly have a good alternative anyway.

Re: alpine version: I think it might be better to not pin to 3.17? What do you think?

That's great! Really good to hear it worked for you as well.
I think having a separate Dockerfile and docker-compose is still standard, at least in my experience with Kubernetes.
Re Alpine 3.17, I think the advantage there is that it comes with OpenSSL 3x instead of OpenSSL 1.1:
3.16
3.17
@jkomyno please correct me if I'm wrong there, be also good to get your opinion here.

I definitely agree with 3.17. My question is whether we want to pin to 3.17 or not pin at all, ie automatically use 3.18 once that is released (seems to be a ~6 month version schedule). My general opinion is that if a new version breaks something I'd rather know this sooner than later, but I can also see an argument for pinning to a known working version.

Also sorry for editing your post, I clicked the wrong thing and thought I was quoting it 😅 should be back to normal now

I have to say, I do like the way you think!

On one end, I would say that pinning the version is in line with "best practices", and that we can trust the end user to make the right call, so we don't have to come back to this every 6 months.

On the other hand, we are pinning the Node version to 16 following similar concerns, so I would say node:16-alpine3.17 makes more sense than node:16-alpine.

Another option, seeing as calling out to apk is no longer a requirement, would be to leave it completely open to the user, and our Dockerfile could look like:

ARG BASE_IMAGE

### BUILDER
FROM --platform=linux/amd64 ${BASE_IMAGE} AS deps

WORKDIR /app

# etc...

That way we are explicitly tasking the user with choosing their distribution and Node version.

[c-ehrlich]

This seems great! I just got it running locally (M1 Pro) and can confirm it works. I benchmarked the Docker x64 emulation a while ago and the overhead was something like 5-10% depending on the task. Given how powerful the M1 is I think this is ok for now, and we don't exactly have a good alternative anyway.

Re: alpine version: I think it might be better to not pin to 3.17? What do you think?

That's great! Really good to hear it worked for you as well.
I think having a separate Dockerfile and docker-compose is still standard, at least in my experience with Kubernetes.
Re Alpine 3.17, I think the advantage there is that it comes with OpenSSL 3x instead of OpenSSL 1.1:
3.16
3.17
@jkomyno please correct me if I'm wrong there, be also good to get your opinion here.

I definitely agree with 3.17. My question is whether we want to pin to 3.17 or not pin at all, ie automatically use 3.18 once that is released (seems to be a ~6 month version schedule). My general opinion is that if a new version breaks something I'd rather know this sooner than later, but I can also see an argument for pinning to a known working version.
Also sorry for editing your post, I clicked the wrong thing and thought I was quoting it 😅 should be back to normal now

I have to say, I do like the way you think!

On one end, I would say that pinning the version is in line with "best practices", and that we can trust the end user to make the right call, so we don't have to come back to this every 6 months.

On the other hand, we are pinning the Node version to 16 following similar concerns, so I would say node:16-alpine3.17 makes more sense than node:16-alpine.

Another option, seeing as calling out to apk is no longer a requirement, would be to leave it completely open to the user, and our Dockerfile could look like:

ARG BASE_IMAGE

### BUILDER
FROM --platform=linux/amd64 ${BASE_IMAGE} AS deps

WORKDIR /app

# etc...

That way we are explicitly tasking the user with choosing their distribution and Node version.

Sorry for the late reply, been busy + jetlagged. Thought about it and agree, pinning is fine.

I think defaulting to users providing their own base image will cause more support requests than anything else 😅

btw i can't remember if there is any particular reason why we're still on Node 16 other than the Docker docs being undermaintained

[nicolomaioli]

No worries! That all sounds good. I'll take Node 18 Alpine 3.17 for a spin and if everything seems to work well I'll get a PR going in the next few days.

I think last time it seemed to work well with updating the code and the English docs leaving the translations untouched, as I will be removing some of the outdated notes. Let me know if that all sounds good with you :)

[c-ehrlich]

No worries! That all sounds good. I'll take Node 18 Alpine 3.17 for a spin and if everything seems to work well I'll get a PR going in the next few days.

I think last time it seemed to work well with updating the code and the English docs leaving the translations untouched, as I will be removing some of the outdated notes. Let me know if that all sounds good with you :)

Sounds good :)

Btw I got a full setup working great on node 18 / alpine 3.17 including next-auth, see here:
#992 (comment)
I based it on an upcoming change to the env schema validation, so maybe makes sense to get that merged before - my implementation of getting next-auth in GSSP to not crash the build by branching in the env.mjs file is a bit sloppy.

Still haven't had time to work on creating a good docker dev setup. With the setup above you can run the app non-dockerized in dev without having to change any files, but I know many projects might want their T3 app to be part of a multi-container setup because that's what they're already on, docker internal networking, etc.

[nicolomaioli]

I'm not sure if this is what you are looking for, but I had a setup for this back when I was deploying to Google Cloud Run (have since move to Vercel). I put together a repo here: t3-docker-compose.

This leverages extended services with two different docker-compose files so you can:

• Spin up a mysql while running the application on the host with npm run dev;
• Spin up a production build of the application alongside a mysql database in a single docker compose stack.

Getting HMR working in docker is a bit of a faff, easiest way is probably Visual Studio Code Dev Containers, for which I have provided a bare-bones implementation.

Also, happy to wait for the other PR to drop, let me know what you think @c-ehrlich

[Mistic92]

We are building our app with kaniko 1.9.1 and even when using Prisma 4.8.0 with gcr.io/distroless/nodejs18-debian11 we still get an error

Command failed with exit code 127: /app/packages/api/node_modules/.prisma/client/query-engine-debian-openssl-1.1.x --enable-raw-queries cli dmmf
the-platform-api-svc-1  | /app/packages/api/node_modules/.prisma/client/query-engine-debian-openssl-1.1.x: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory