Bad credentials when downloading using @prisma/fetch-engines

[rococtz]

Bug description

I am using Prisma as part of AWS Lambda deployed with serverless. When packaging the project, we have a custom step (as a javascript file) which downloads prisma engines. Starting today, this step doesn't work anymore because of bad credentials from the CDN hosting the engines.

How to reproduce

const { download } = require('@prisma/fetch-engine');

await download({
  binaries: {
    'migration-engine': resolve('./node_modules/@prisma/engines/'),
  },
  binaryTargets: ['rhel-openssl-1.0.x'],
  ignoreCache: !0,
  printVersion: !0,
  failSilent: !1,
  showProgress: !0,
  progressCb: (e, n) => {...},
})

The errors that I get:

{
  message: 'Bad credentials',
  documentation_url: 'https://docs.github.com/rest'
}

from:

.../node_modules/@prisma/fetch-engine/dist/getLatestTag.js:54
    throw new Error(`Could not fetch commits from github: ${JSON.stringify(commits, null, 2)}`);

URL which cannot be reached:
https://github-cache.prisma.workers.dev/repos/prisma/prisma-engines/commits?sha=${branch}

Expected behavior

No response

Prisma information

my package.json:

"@prisma/client": "^4.7.1"
"prisma": "^4.7.1"
"@prisma/fetch-engine": "^4.7.1"

Environment & setup

• OS: Ubuntu
• Database: Postgres
• Node.js version: 16.15.1

Prisma Version

prisma                  : 4.7.1
@prisma/client          : 4.7.1
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine 272861e07ab64f234d3ffc4094e32bd61775599c (at node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
Migration Engine        : migration-engine-cli 272861e07ab64f234d3ffc4094e32bd61775599c (at node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Introspection Engine    : introspection-core 272861e07ab64f234d3ffc4094e32bd61775599c (at node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x)
Format Binary           : prisma-fmt 272861e07ab64f234d3ffc4094e32bd61775599c (at node_modules/@prisma/engines/prisma-fmt-debian-openssl-1.1.x)
Format Wasm             : @prisma/prisma-fmt-wasm 4.7.1-1.272861e07ab64f234d3ffc4094e32bd61775599c
Default Engines Hash    : 272861e07ab64f234d3ffc4094e32bd61775599c
Studio                  : 0.477.0
Preview Features        : interactiveTransactions

[janpio]

Investigating. This should now be fixed.

[Jolg42]

Indeed, it should be fixed, It was due to an expired GitHub token on that worker.

[janpio]

You should be able to prevent on this supposedly internal web service by providing an explicit version to the function. Only when non is provided it assumes latest and then talks to this API to get the latest version. That is most probably a thing you do not want your build pipeline to rely on anyway - it might not match the Prisma version of your app you are building and hence could introduce subtle bugs.

Can you share what exactly you are doing in that serverless build pipeline? That sounds interesting to understand for us. This code was definitely not written for that use case :D

[rococtz]

@janpio @Jolg42 Thank you for responding, I will be able to test again in several hours.

First of all, I would like to ask you if there's any recommendation on how to stick to downloading fixed version.

The way we use it is basically like this:

• serverless packager calls custom script
• custom script downloads these engines so they can be packaged
• the code gets deployed to AWS Lambda which is then invoked to run/apply migrations on the DB

Is there any better/recommended way to package these engines into an AWS Lambda which has only one purpose which is running the migrations?

[janpio]

latest, or not providing a version, is actually not fixed but will change when we release a new version. You should provide an explicit version there if possible.

The "normal" way to deal with your problem (that you want to have rhel-openssl-1.0.x engine files when developing somewhere else) is probably to use binaryTargets in the generator block of your Prisma schema, and then run a command locally that downloads the appropriate query engine - like prisma generate.

As you mention that you need the migration engine though, a command to trigger that download would be for example prisma -v as that will make sure all of them are present. There the binaryTargets of the schema are not used though. So we have this handy environment variable: https://www.prisma.io/docs/reference/api-reference/environment-variables-reference#cli-binary-targets The example shows this already influencing npm install, so maybe you do not even have to call any command.

Does this help?

[rococtz]

@janpio Thank you, this is helpful 👍