Make engineType configurable via an environment variable

[matthewmueller]

Problem

The engine type should change depending on if you're working locally or in production. The typical way to do this is to use environment variables. However, right now it's not possible to change the engine type based on environment.

This means that when you deploy to different environments, you need to manually edit your Prisma Schema. This is error prone.

Suggested solution

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

generator client {
  provider        = "prisma-client-js"
  engineType      = env("ENGINE_TYPE")
  previewFeatures = ["dataproxy"]
}

Alternatives

Specify a CLI flag, something like:

prisma generate --engine-type=dataproxy

[janpio]

Setting the engineType value on the already generated Client will not work, as this only influences the generation process of the generator block. The expectation of an env var in a datasource block, (see env("DATABASE_URL")), on the other hand is that this indeed does influence the already generated Client. So if we use an env var for engineType, the expectation might naturally be the same. Problem is, that the engine type dataproxy can only be set during Client generation, as it has influence on the included files.

Additional complexity comes from the fact that there already is a general env var to override the engineType value for the generated Client (and during Client generation): https://www.prisma.io/docs/reference/api-reference/environment-variables-reference#prisma_client_engine_type

engineType was not designed to be used the way this is probably necessary in this case.

[seth]

Reading over this along with recent discussion in Slack...

The generator config seems to have a different lifecycle than the PSL. Part of what's cool about PSL is you could (and often do) generate different clients with different options. To me it looks like we're using the PSL as a convenient place to specify and transfer config for the (our) generator. I like that it keeps things to a single file (convenient), but here we're seeing the lack of flexibility mismatch in lifecycle.

It doesn't look like prisma generate takes config on the CLI (?) but it could. And generate could look for a file. The downside is now there's another project config file to manage. The upside is you could have generate.prisma.prod and generate.prisma.local

[albertoperdomo]

I think a CLI is a sensitive option that would work and make things simpler:

Env var:

• Need to have a line in the schema engineType=env("ENGINE_TYPE")
• More friction in setup: need one more env var in each deployment and in the local dev environment, 3 in total (DB URL, Proxy URL, Engine Type)
• Could cause confusion to users leading them to think they can change the engine type at runtime, as hinted by @janpio - though this setting only has an impact at generation time

CLI param:

• The line to engineType in the schema file can go away - this is good for folks who don’t plan to use the proxy at all (they didn’t need it, once less difference between using/not using the proxy) and in general (fewer LO code/config in the schema file)
• One less env var in the deployment and in the local dev environment, one less change required to start/stop using the proxy
• The engine param can be pre-set for our users in the build task in package.json we make part of our templates, so this step is a no-op for them. Locally it would default to whatever the default enginetype is set to, e.g. "vercel-build": "prisma generate --engineType=proxy && …", See
• For other deployment platforms we don't have templates/examples for, we can include this step in our deployment guidance section of the proxy docs and guidance displayed in the cloud UI

I suggest we explore the CLI param option, since it seems to not cause confusion potentially and seems to cause less friction for users to start/stop using the Prisma Data Proxy and everyone else who doesn’t need to set the engine type to a custom value.

[dan-made]

More friction in setup: need one more env var in each deployment and in the local dev environment, 3 in total (DB URL, Proxy URL, Engine Type)

Howdy, I just started deploying my first data-proxy example and I'm wrestling with this exact issue. I would say from a dev perspective this doesn't sound especially burdensome? If I'm doing it right (big if!) the env helper sounds like what I want, and being responsible for an env variable is what I would expect to be the cost.

[janpio]

There happened internal discussion around this that is not replicated here - but we are honing in on a solution.

If you have the time to experiment, there actually might already be one implemented to try out and report back if it works:
Set the environment variable PRISMA_CLIENT_ENGINE_TYPE (https://www.prisma.io/docs/reference/api-reference/environment-variables-reference#prisma_client_engine_type) when generating your production client that should use the Data Proxy together with your specific Data Proxy connection string, and do not set the engineType = "dataproxy" in the schema file. This should make the development version use the local engine, but generated a Client for the data proxy for production (via the env var being set).

[matthewmueller]

Adding to @janpio's comment, generating a data proxy-compatible client would look like this in Preview:

1. Enable the dataproxy preview feature in your Prisma Schema.

   generator client {
     provider        = "prisma-client-js"
     previewFeatures = ["dataproxy"]
   }

2. Run PRISMA_CLIENT_ENGINE_TYPE=dataproxy npx prisma generate

Questions about the interplay:

• What are the interactions between the preview feature and the engine type? What happens if you specify one but not the other?
• What happens if the preview feature has the dataproxy set, but the engine type is library?

[dan-made]

@janpio this is available on 3.1.0-integration-data-proxy-engine.1 or should I upgrade to a newer release?

[janpio]

@dan-made that should be available in that version, yes. But I am trying this out myself right now as well, so feel free to wait until I report back.

[dan-made]

A quick test deployment failed with the URL must start with the protocol postgresql://orpostgres://`` and I don't have a new enough version to add dataproxy to preview features, so I will pump the brakes.

Either way this sounds better than editing the schema file in CI, which was my plan an hour ago 😅

[janpio]

Will report back in a bit where we are and let you know.

(Dataproxy preview features is not a thing yet, @matthewmueller is talking about the future setup here - so that is expected.)

[dan-made]

If it is at all relevant here, I get the URL must start with build error even when I have engineType hardcoded in my schema.

I had db push in my build script for testing it out and it occurs to me this is probably the same issue surfaced about migrate. Removing that script builds as expected with the env variable set.

[janpio]

I can confirm that PRISMA_CLIENT_ENGINE_TYPE can be used to override whatever is or is not set as the engineType in the schema file already.

[matthewmueller]

Closing since this is already possible with PRISMA_CLIENT_ENGINE_TYPE