chore(client): [DPGA] implement --data-proxy #13484

millsp · 2022-05-25T10:33:47Z

This PR replaces the PRISMA_CLIENT_ENGINE_TYPE=dataproxy mechanism in favor of --data-proxy via the cli.

https://github.com/orgs/prisma/projects/70#card-82299232

SevInf · 2022-05-25T13:08:01Z

Warning: potentially stupid question ahead.
Should we do something in case when engineType defined explicitly and --data-proxy flag is used? I am not sure what hypothetical user wants to achieve with this combination, but engineType would be ignored in this case, so maybe it worth a warning?

millsp · 2022-05-25T17:01:44Z

It's a good question, so I'll try to answer with what I think is my best opinion.

If we stick to the fact that --data-proxy means "I can only use Client with the Data Proxy from now",
as I understand it, we don't need to emit a warning since the intent of the user is clearly expressed.
If --data-proxy is enabled, they will get an error in any case where the datasource is not prisma://.
And alternatively, they will also get an error if they pass a prisma:// url and --data-proxy isn't set.
Also, engineType is a user-facing implementation detail meant to be an escape hatch (not a feature).

millsp · 2022-05-27T10:31:39Z

packages/client/fixtures/dataproxy/prisma/schema.prisma

@@ -5,7 +5,6 @@ datasource db {

 generator client {
  provider   = "prisma-client-js"
-  engineType = "dataproxy"


Not a valid entry anymore.

millsp · 2022-05-27T10:32:06Z

packages/client/src/__tests__/buildNFTAnnotations.test.ts

@@ -11,7 +11,7 @@ function normalizePaths(snapshot: string): string {

 describe('library', () => {
  it('generates annotations for a schema and a single engine', () => {
-    const annotations = buildNFTAnnotations(ClientEngineType.Library, ['debian-openssl-1.1.x'], 'out')
+    const annotations = buildNFTAnnotations(false, ClientEngineType.Library, ['debian-openssl-1.1.x'], 'out')


dataProxy = false

millsp · 2022-05-27T10:32:38Z

packages/client/src/__tests__/buildNFTAnnotations.test.ts

@@ -84,16 +86,15 @@ describe('binary', () => {
 describe('dataproxy', () => {
  it('generates no annotations', () => {
    const annotations = buildNFTAnnotations(
-      ClientEngineType.DataProxy,
+      true,


dataProxy = true

millsp · 2022-05-27T10:33:23Z

packages/client/src/__tests__/integration/happy/custom-engine-binary/test.ts

@@ -5,11 +5,6 @@ import path from 'path'

 import { generateTestClient } from '../../../../utils/getTestClient'

-if (getClientEngineType() === ClientEngineType.DataProxy) {


Not needed.

millsp · 2022-05-27T10:34:14Z

packages/client/src/generation/TSClient/TSClient.ts

@@ -86,15 +88,16 @@ export class TSClient implements Generatable {
      engineVersion: this.options.engineVersion,
      datasourceNames: datasources.map((d) => d.name),
      activeProvider: this.options.activeProvider,
+      dataProxy: this.options.dataProxy,


dataProxy is now stored in the generated config instead of being in the generator config block (via schema).

millsp · 2022-05-27T10:34:29Z

packages/client/src/generation/TSClient/TSClient.ts

@@ -62,6 +63,7 @@ export class TSClient implements Generatable {
      runtimeDir,
      runtimeName,
      datasources,
+      dataProxy,


Passed by the generator.

millsp · 2022-05-27T10:34:53Z

packages/client/src/generation/TSClient/TSClient.ts

-${buildWarnEnvConflicts(engineType, runtimeDir, runtimeName)}
+${buildInlineDatasource(dataProxy, datasources)}
+${await buildInlineSchema(dataProxy, schemaPath)}
+${buildInlineEnv(dataProxy, datasources, envPaths)}


This will be removed in the followup PR.

millsp · 2022-05-27T10:36:14Z

packages/client/src/generation/TSClient/TSClient.ts

    }

    // get relative output dir for it to be preserved even after bundling, or
    // being moved around as long as we keep the same project dir structure.
    const relativeOutdir = path.relative(process.cwd(), outputDir)

    const code = `${commonCodeJS({ ...this.options, browser: false })}
-${buildRequirePath(engineType)}
-${buildDirname(engineType, relativeOutdir, runtimeDir)}
+${buildRequirePath(dataProxy)}


We don't use the engine type in most cases anymore as that was used to do the data proxy distinction at generation time.

millsp · 2022-05-27T10:36:36Z

packages/client/src/generation/generateClient.ts

@@ -57,7 +58,6 @@ export interface BuildClientResult {

 // eslint-disable-next-line @typescript-eslint/require-await
 export async function buildClient({
-  datamodel,


millsp · 2022-05-27T10:37:19Z

packages/client/src/generation/generateClient.ts

@@ -48,6 +48,7 @@ export interface GenerateClientOptions {
  engineVersion: string
  clientVersion: string
  activeProvider: string
+  dataProxy: boolean


Passed to generateClient via the generator.

millsp · 2022-05-27T10:47:10Z

packages/client/src/generation/generateClient.ts

@@ -238,7 +242,8 @@ export async function generateClient({
      }\` in the \`binaryPaths\` object.`,
    )
  }
-  if (transpile) {
+
+  if (transpile === true && dataProxy === false) {


We don't copy the query engine files if the data proxy is in use.

millsp · 2022-05-27T10:47:44Z

packages/client/src/generation/generator.ts

@@ -51,6 +51,7 @@ if (process.argv[1] === __filename) {
        clientVersion,
        transpile: true,
        activeProvider: options.datasources[0]?.activeProvider,
+        dataProxy: options.dataProxy,


Passed by the cli.

millsp · 2022-05-27T10:50:11Z

packages/client/src/runtime/getPrismaClient.ts

@@ -449,7 +458,9 @@ export function getPrismaClient(config: GetPrismaClientConfig) {
      return 'PrismaClient'
    }
    private getEngine(): Engine {
-      if (this._clientEngineType === ClientEngineType.Library) {
+      if (this._dataProxy === true) {


dataProxy = true is an override of the other engines, as explicitly asked via --data-proxy

millsp · 2022-05-27T10:51:01Z

packages/client/tests/functional/typescript/tests.ts

@@ -36,7 +36,7 @@ function getAllTestSuiteTypeChecks(fileNames: string[]) {

 describe('typescript', () => {
  const suitePaths = glob.sync('./**/.generated/**/*.ts', {
-    ignore: ['./**/.generated/**/*.d.ts', './**/.generated/**/_schema.ts'],
+    ignore: ['./**/.generated/**/*.d.ts', './**/.generated/**/_*.ts'],


Unrelated change.

millsp · 2022-05-27T10:51:16Z

packages/debug/helpers/build.ts

@@ -1,3 +1,3 @@
 import { build } from '../../../helpers/compile/build'

-void build([{ name: 'default' }])
+void build([{ name: 'debug' }])


Unrelated change.

millsp · 2022-05-27T10:51:53Z

packages/engine-core/src/data-proxy/DataProxyEngine.ts

@@ -237,7 +234,7 @@ export class DataProxyEngine extends Engine {
    const { protocol, host, searchParams } = url

    if (protocol !== 'prisma:') {
-      throw new InvalidDatasourceError('Datasource URL should use prisma:// protocol. If you are not using the Data Proxy, remove the `dataProxy` from the `previewFeatures` in your schema and ensure that `PRISMA_CLIENT_ENGINE_TYPE` environment variable is not set to `dataproxy`.', {
+      throw new InvalidDatasourceError('Datasource URL should use prisma:// protocol when --data-proxy is used', {


TODO: use "must" instead of "should"

millsp · 2022-05-27T10:53:07Z

packages/engine-core/src/data-proxy/utils/request.ts

-  },
-  RequestInit
->
+// our implementation handles less than the real fetch


Because of the removal of ambient types from webworkers.

millsp · 2022-05-27T10:54:16Z

packages/migrate/src/Migrate.ts

@@ -155,6 +155,7 @@ export class Migrate {
      printDownloadProgress: true,
      version: enginesVersion,
      cliVersion: packageJson.version,
+      dataProxy: false,


We don't allow --data-proxy on prisma migrate as they are not currently compatible.

millsp · 2022-05-27T10:55:41Z

packages/sdk/src/__tests__/getGenerators/getGenerators.test.ts

-            provider = \\"prisma-client-js\\"
-            previewFeatures = [\\"dataProxy\\"]
-        }
+        interactiveTransactions preview feature is not yet available with --data-proxy.


TODO: re-enable this test.

millsp · 2022-05-27T10:56:17Z

packages/sdk/src/client/getClientEngineType.ts

@@ -3,7 +3,6 @@ import type { GeneratorConfig } from '@prisma/generator-helper'
 export enum ClientEngineType {
  Library = 'library',
  Binary = 'binary',
-  DataProxy = 'dataproxy',


Not allowed here anymore.

millsp · 2022-05-27T10:57:33Z

packages/sdk/src/get-generators/utils/check-feature-flags/checkFeatureFlags.ts

 }

-function checkProxyFeatureFlag(config: ConfigMetaFormat) {


Not needed anymore.

millsp · 2022-05-27T10:57:57Z

packages/sdk/src/get-generators/utils/check-feature-flags/checkFeatureFlags.ts

 }

-function checkProxyFeatureFlag(config: ConfigMetaFormat) {
+function checkForbiddenItxWithDataProxyFlag(config: ConfigMetaFormat, options: GetGeneratorOptions) {


Make sure that itx cannot be used with --data-proxy.

millsp · 2022-05-30T20:09:29Z

Replaced by #13561

millsp added 4 commits May 24, 2022 20:12

fix(lint): failing on sub-projects and simplify it

62a09ae

chore(deps): bump typescript

b332e65

chore: update locks

245e81f

feat(client, prisma): enable the data proxy via --data-proxy

e2db143

millsp changed the title ~~fix(lint): failing on sub-projects and simplify it~~ chore(client): Data Proxy GA May 25, 2022

millsp added 3 commits May 25, 2022 13:19

skip engine copying when --data-proxy is passed

a4dacab

finish fix webworker type leakage

4093ab3

pass dataProxy boolean in missing places

c66bda1

millsp added this to the 3.15.0 milestone May 25, 2022

millsp self-assigned this May 25, 2022

Merge branch 'main' of github.com:prisma/prisma into chore/data-proxy-ga

4ace46f

fix sdk compiling test types

47f2002

millsp added 3 commits May 26, 2022 14:44

chore(build): fix watch on delete file

3954378

fix missing dataProxy parameter option

b8b50e5

remove unnecessary tsconfig.json

355dc27

millsp changed the title ~~chore(client): Data Proxy GA~~ chore(client): [DPGA] implement --data-proxy May 26, 2022

fix serializing array buffer view

7be0448

millsp mentioned this pull request May 26, 2022

chore(client): split the runtime into two [DPGA, 2] #13512

Merged

millsp added 7 commits May 26, 2022 19:29

fix generateInFolder missing dataProxy option

0376135

remove last instances of data proxy engine type

de80079

remove data proxy engine type in tests

d575b57

fix missing dataproxy arguments

3c393c7

update snaps

bf62a31

make linting more performant

77a6e71

attempt to fix linting and type performance

5fa27f8

millsp force-pushed the chore/data-proxy-ga branch from 587c573 to 5fa27f8 Compare May 26, 2022 22:22

millsp added 2 commits May 27, 2022 01:12

undo linting changes & ts 4.7

c67fb85

revert linting related changes

4b0a0b8

millsp commented May 27, 2022

View reviewed changes

millsp added 5 commits May 27, 2022 13:37

fix types on inlineDatasources

7d9db1c

bump typescript

78240b0

update linting

01d2823

Merge branch 'chore/ts-4.7' into chore/data-proxy-ga

f530901

fix types

ae6f4cc

millsp closed this May 30, 2022

Jolg42 deleted the chore/data-proxy-ga branch July 5, 2022 07:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(client): [DPGA] implement --data-proxy #13484

chore(client): [DPGA] implement --data-proxy #13484

millsp commented May 25, 2022 •

edited

SevInf commented May 25, 2022

millsp commented May 25, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp May 27, 2022

millsp commented May 30, 2022

		@@ -5,11 +5,6 @@ import path from 'path'

		import { generateTestClient } from '../../../../utils/getTestClient'

		if (getClientEngineType() === ClientEngineType.DataProxy) {

chore(client): [DPGA] implement --data-proxy #13484

chore(client): [DPGA] implement --data-proxy #13484

Conversation

millsp commented May 25, 2022 • edited

SevInf commented May 25, 2022

millsp commented May 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

millsp commented May 30, 2022

millsp commented May 25, 2022 •

edited