Deploying Prisma with AWS Lambda and Webpack #2303
Comments
|
As is typical, all roads lead back to webpack! I was finally able to get this working, although not exactly on windows (see below), although it does work with standard WSL. The main issue was serverless webpack running another yarn install prior to packaging, which was not running my prisma generate properly. This caused my @prisma/client to be either empty, or at least not include the required binaries. Moral of the story, you cannot simply let webpack package @prisma/client as it will roll in all the js, but not keep any binaries around. Here is my final setup: Serverless.aml plugins:
- serverless-webpack
provider:
name: aws
custom:
webpack:
webpackConfig: 'webpack.config.js'
includeModules: true # This is required
packager: 'yarn'
packagerOptions:
scripts: # this is the magic
- prisma generate webpack.config.js (standard serverless webpack stuff excluded) const CopyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
externals: [nodeExternals()], // this is required
plugins: [
new CopyWebpackPlugin(['./schema.prisma']), // without this the prisma generate above will not work
]
};prisma.schema generator client {
provider = "prisma-client-js"
binaryTargets = ["native", "rhel-openssl-1.0.x"]
}With these three things I was able to build and deploy the serverless api from osx while still using serverless-webpack. The real magic is the copy of the schema file, and running the prisma generate script post build. Unfortunately I cannot deploy from windows currently as when I run primsa generate with the above schema it simply hangs forever. When I remove rhel-oplenssl-1.0x from the binaryTargets it succeeds, but only generates the windows binary. Ideally on windows it would be nice to be able to generate whichever binary is specified, even though it can't be ran. I hope this helps someone in the future! I am going to leave this open for now, at the discretion of the owners, as I still cannot deploy from windows, but the bulk of the issue (deploying prisma2 using serverless, serverless webpack, to aws lambda) has been solved. |
|
You are a hero bruv! 🔥 |
|
@dested Thanks!! 🙌 |
|
I'm getting the following error |
|
If you put SLS_DEBUG=* into your path itll give you the expanded error, but more than likely its due to prisma not being found. The only way I was able to get around this was making sure prisma cli was installed globally |
|
@dested installing |
|
@dested Thanks for the great write up. You are using webpack to bundle your application and you need to copy the binaries over. In your case, you are using Prisma generate via webpack to do so, while that would also work, the faster way to do it will be to copy the binaries directly. This is where we touch on that in the docs https://www.prisma.io/docs/reference/tools-and-interfaces/prisma-client/module-bundlers Does that make sense? Please feel free to ask further questions. P.S. I edited the issue title to reflect its state better, please let me know if you do not agree with it. |
divyenduz
changed the title
|
@divyenduz, the new title is perfect. So there are a couple of issues that I ran into along the way that makes simply using webpack copy insufficient, specifically that serverless-webpack runs its own On the note about simply copying over the binaryTargets for other platforms when on windows, it very strangely just fails silently. Because of this, to deploy I must switch to WSL, which is more annoying than anything. I will post that as a separate issue later today after I try to track down exactly what is failing. Thanks for your support! |
|
thank you very much @dested 🔥👍 |
I have a one more question. @dested I am getting this following error below How can I do ? This is my webpack.config.js const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const CopyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
entry: slsw.lib.entries,
target: 'node',
// Generate sourcemaps for proper error messages
devtool: 'source-map',
// Since 'aws-sdk' is not compatible with webpack,
// we exclude all node dependencies
externals: [nodeExternals()],
mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
optimization: {
// We do not want to minimize our code.
minimize: false,
},
performance: {
// Turn off size warnings for entry points
hints: false,
},
plugins: [
new CopyWebpackPlugin(['./prisma/schema.prisma']), // without this the prisma generate above will not work
],
// Run babel on all .js files and skip those in node_modules
module: {
rules: [
{
test: /\.js$/,
loader: 'babel-loader',
include: __dirname,
exclude: /node_modules/,
},
],
},
}; |
|
I solved
I solved my issue plugins: [
new CopyWebpackPlugin({ patterns: ['./prisma/schema.prisma'] }), // without this the prisma generate above will not work
], |
Query engine binary for current platform "darwin" could not be found.
This probably happens, because you built Prisma Client on a different platform.
(Prisma Client looked in "/query-engine-darwin")
Files in /:Why in the heck is Prisma Client looking at the top path of my freaking 💻 ? 🗯️ I am configuring in plugins: [
new CopyWebpackPlugin({
patterns: [
{
from: path.join(
__dirname,
"node_modules/@prisma/cli/query-engine-darwin"
),
// 'dist' is just the 'webpack' default
to: path.join(__dirname, "dist"),
},
],
}),
],It doesn't matter whether it copies or not because the code from 'dist/main.js' seems to always look 👀 at the top-level absolute path on my 💻 . Why???????? 😱 |
|
Does someone have a working boilerplate of this somewhere I could inspect. Im still struggling at bit. Or could maybe spot something I'm doing wrong.... webpack config serverless.yml dependencies And then the build on our CI server fails with an error like this. So something clearly wrong with my webpack config, or maybe the versions of dependencies are out of sync? |
|
@sheldonj Here is a project I did early on in the year for you to look at: https://github.com/AmoDinho/prisma2-f1-demo This is what my Webpack config looks like: |
|
My previous issue is solved! I didn't realize about const nodeExternals = require("webpack-node-externals");
module.exports = {
mode: "development",
target: "node",
node: {
__dirname: true,
},
externals: [nodeExternals()],
module: {
rules: [
{ test: /\.graphql?$/, loader: "webpack-graphql-loader" },
{
test: /\.js$/,
use: {
loader: "babel-loader",
options: {
presets: [["@babel/preset-env", { targets: { node: true } }]],
},
},
},
],
},
resolve: { modules: ["src", "node_modules"] },
}; |
|
I encounter the same problem...using
with the above config, I am able to see the |
|
I was able to bundle a vanilla app with prisma for Lambda with this webpack config without using /// <reference path="./typings/webpack-permissions-plugin/index.d.ts" />
import glob from 'glob'
import path from 'path'
import { Configuration, IgnorePlugin } from 'webpack'
import { CleanWebpackPlugin } from 'clean-webpack-plugin'
import CopyPlugin from 'copy-webpack-plugin'
import PermissionsPlugin from 'webpack-permissions-plugin'
interface EntryOutput {
entry: string
output: {
filename: string
path: string
}
}
const entries: EntryOutput[] = glob
.sync('./src/handlers/*-handler.ts')
.reduce((prev: EntryOutput[], curr: string): EntryOutput[] => {
const filename = path.basename(curr, '.ts')
return [
...prev,
{
entry: path.resolve(__dirname, curr),
output: {
filename: `${filename}.js`,
path: path.resolve(__dirname, 'dist', filename),
},
},
]
}, [])
const config: Configuration[] = entries.map(({ entry, output }) => ({
entry,
output: {
...output,
libraryTarget: 'commonjs',
},
target: 'node',
externals: {
'aws-sdk': 'commonjs2 aws-sdk',
},
devtool: 'source-map',
node: false,
mode: 'production',
resolve: {
extensions: ['.ts', '.js', '.json'],
},
module: {
rules: [
{
test: /\.ts$/,
use: 'ts-loader',
exclude: /node_modules/,
},
],
},
plugins: [
new CleanWebpackPlugin(),
new IgnorePlugin({
resourceRegExp: /encoding/,
}),
new CopyPlugin({
patterns: [
{
from: path.resolve(
__dirname,
'./node_modules/.prisma/client/query-engine-rhel-openssl-1.0.x'
),
to: output.path,
},
{
from: path.resolve(
__dirname,
'./node_modules/.prisma/client/schema.prisma'
),
to: output.path,
},
],
}),
new PermissionsPlugin({
buildFiles: [
{
path: path.resolve(
__dirname,
`${output.path}/query-engine-rhel-openssl-1.0.x`
),
fileMode: '755',
},
],
}),
],
}))
export default configI have another issue open which I have worked around in this config #5250 |
Did you ever figure this out? I'm getting the same error as you, with a similar setup: .babelrc webpack config serverless.yml |
|
Try to downgrade webpack to 4 version and all it deps, like copy-webpack-plugin to 6.4.0 to support webpack v4 |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as outdated.
This comment was marked as outdated.
|
This was seriously doing my head in! Two days wasted of trying to deploy a TypeScript / Prisma / Serverless app to AWS Lambda.. It really shouldn't be that hard. Problem lies around copying the Prisma client including the binaries into the deployment package which is not all that transparent. In the end this module saved me:
This is the results of my efforts: A working deployable bare-bones AWS Lambda function using TypeScript, Prisma and Amazon Aurora RDS PostgresQL. |
janpio
added
bug/0-unknown
janpio
added
topic: Query engine ... could not be found.
You save me. Thank you so much. |
|
Hi everyone! Just wanted to provide an update in this thread. We don't believe this is a bug in Prisma, per se, but it's also not a feature. Unfortunately the issues we've been seeing with bundlers (webpack, esbuild, etc) is due to files that Prisma needs to function not being copied into resulting bundles. To that end, we are looking to improve our documentation so that future individuals don't have to tread down the same very painful path that y'all have. Our first pass at this is live. Please check out our new documentation on deploying with Serverless Framework and let us know what you think. |
|
Hi there! Here's one approach to address this issue: place the query engine in the same directory as your serverless function file. To do this, follow these steps:
Here is an example with
|
and others
Bug description
I want to start off by saying I am absolutely in love with the prisma platform, and everything it provides, and I can't state enough how much I appreciate the efforts of the entire team. Thank you.
I have spent the better part of a weekend trying in earnest to deploy a simple rest api using prisma2, unsuccessfully for one reason or another. While I am deploying and building on windows (which is obviously an issue with the binary), I have also tried to build and deploy from wsl, as well as github actions (both debian base, while lambda is centos based). I have tried every webpack configuration I could think of, including every possible binary I can scrounge up (for instance, manually copying query-engine-rhel-openssl-1.0.x into the directory), all with zero luck. I have tried outputting prisma client to a directory that is not node_modules, copying that, referencing it, all with no changes. My initial thinking was that because I am building on windows it was causing issues, and while it was causing an issue, not using windows did not solve my problems.
I know there are a handful of issues that address this in bits and pieces, some old, some really old, some about Windows, some about chmod, some about serverless, some about photon, I have been through all of them with no success.
The majority of the time my error is simply:
Needless to say, it's been suboptimal. Before I continue down my journey into madness, my question is simply: is this a valid usecase? Is it possible to have prisma deployed to aws lambda (ideally using the serverless framework)?
How to reproduce
Expected behavior
Query completes successfully
Prisma information
Environment & setup
OS: Windows, WSL, and Ubuntu-latest via github actions
Database: PostgreSQL
Prisma version: @prisma/cli : 2.0.0-beta.3
Current platform : windows
Query Engine : query-engine 2fb8f44
Migration Engine : migration-engine-cli 2fb8f44
Introspection Engine : introspection-core 2fb8f44
Node.js version: v12.16.2
The text was updated successfully, but these errors were encountered: