Requesting Cloud Functions for Firebase to support pnpm workspaces

[christophe-g]

I am facing a pretty serious situation in which I cannot deploy/fix any functions to my project (pnpm based).

Trying to deploy fail with this error:

Build failed: This project is using pnpm but you have not included the Functions Framework in your dependencies. Please add it by running: 'pnpm add @google-cloud/functions-framework'.; Error ID: 5b6dc8b5

Functions deploy had errors with the following functions:
	bigben(us-central1)
i  functions: cleaning up build files...

Error: There was an error deploying functions

The proposed fix (adding @google-cloud/functions-framework) does not help.

[REQUIRED] Environment info

firebase-tools: 12.2.1
Platform: Linux Mint 21.1

[REQUIRED] Test case

• create a new project on firebase console
• enable functions
• follow the getting started guide (https://firebase.google.com/docs/functions/get-started?gen=1st) , installing dependencies with pnpm in root directory and functions directory
• deploy a new function (firebase deploy --only functions:bigben)

The log file for the deploy operation is attached.

[REQUIRED] Steps to reproduce

See above

[REQUIRED] Expected behavior

Functions should deploy when a project is set-up with pnpm

[REQUIRED] Actual behavior

See error above:

• existing functions do not update
• new functions no not deploy

I am not sure since when this occurs, but I deployed new functions without any issues 3rd May.

firebase-debug.log

[christophe-g]

Hmmm, removing pnpm-lock.yaml in the functions source folder results in a successful deployment. Not too sure what to make of it.

[inlined]

Firebase does not officially support pnpm. In order to deploy your functions, we need to load your code using a script we install with the firebase-functions SDK. Pnpm workspaces make it hard to do this sort of bootstrapping of your code.

[christophe-g]

I have been using pnpm with firebase functions since a couple of years without any problem (I am not using workspace - just pnpm as the dependency manager).

The fact that functions deploy properly when I remove pnpm-lock.yaml smells a false positive check somewhere.

[christophe-g]

@taeold - can this be retriaged to bug as the success of the deploment relies on the existence ot pnpm.lock.yaml only (e.g. I can deply when the file does not exist, and inversely).

[angelcervera]

Firebase does not officially support pnpm. In order to deploy your functions, we need to load your code using a script we install with the firebase-functions SDK. Pnpm workspaces make it hard to do this sort of bootstrapping of your code.

So for sure, this information should be, at least, in the documentation. It is a huge limitation. This one, and also the one related to workspaces.

[m-shaka]

Note: If a package-lock.json, yarn.lock, or pnpm-lock.yaml file is found within your project, that lock file will be respected when dependencies are installed using npm ci, yarn install or pnpm install.

https://firebase.google.com/docs/functions/handle-dependencies

Doesn't this document mean that Firebase supports pnpm?

I've encountered the exact same problem and removing pnpm-lock.yaml fixed it for me too.
I confirmed that without pnpm-lock.yaml firebase CLI used npm instead of pnpm, that is the behavior described in the document.

[m-shaka]

@christophe-g
FYI: For me adding @google-cloud/functions-framework as root dependencies worked, while adding it as devDependencies or workspace package's dependencies didn't. I have no idea why functions-framework is needed.

I deployed 1st gen functions using pnpm workspace

[christophe-g]

@colerogers - any chance to have this reviewed and fixed ?

Please note that this is not a feature request - but a bug as it was previously working as expected (possible to deploy with pnpm as stated in the documentation).

Thanks.

[colerogers]

Hi @christophe-g apologies for the delayed response. So I did a little digging on this issue and the error is not actually coming from the Firebase CLI, but instead the Cloud Build job.

Since a Firebase function is actually a Google Cloud Function under the hood, we send your source code over to Cloud Build and that job uses Buildpacks to build source to a container image.

I dug a little bit deeper and found that around the time that you started seeing this behavior (end of may), the buildpacks team published support for pnpm. They state that you would need to have the @google-cloud/functions-framework dependency installed, like the error gives.

Since it sounds like you still have this issue with the dependency installed, I think opening a bug on their repo would be a good next step. I'm sorry that this has taken so long and that you have to jump through more hoops just to get this resolved. Thanks

[christophe-g]

Thanks a lot @colerogers - appreciated.

I will try to install @google-cloud/functions-framework again and update all related dep. Will post the results here.

[0x80]

In case you'd like to deploy only an isolated part of your pnpm monorepo to firebase, have a look at this solution It also lets you deploy to firebase from multiple packages.

[johndunderhill]

@0x80 Great work on isolate. Very nice. We'll be trying it in a pnpm monorepo next week. I really appreciate your efforts.

[0x80]

In case you like to avoid adding the @google-cloud/functions-framework dependency, the isolate-package forceNpm configuration lets you output an npm package-lock.json file instead, so you can deploy it as if your code does not depend on pnpm. The lockfile versions should match the pnpm lockfile, and I think that's all that matters really.

[johndunderhill]

Thanks, @0x80.

Do you know about the (possibly undocumented) --ignore-workspace flag for pnpm? Using this I can install dependencies in a subfolder based on a dummy package.json residing there, and generate a new, stand-alone pnpm lockfile, even while still inside the workspace heirarchy.

For example:

#!/bin/bash
# File: deploy-node

if [[ -z $1 ]] || [[ -z $2 ]]; then
	echo "Usage: deploy-node <source-code-directory> <output-directory>"
	echo "Run this command from within the package you wish to deploy"
  	echo "The entry point in the source code directory is assumed to be index.ts"
	echo "Example: deploy-node src dist"
	echo "Warning: The output directory will be emptied!"
	exit 1
fi

if [[ -f $2 ]]; then
	echo "Error: $2 is a file!"
	exit 2
fi

mkdir -p $2
rm -rf $2/*

cp .firebaserc $2
cp firebase.json $2
cp $REPO_ROOT/.nvmrc $2
cp package.template.json $2/package.json
bundle-node $1 $2
cd $2
pnpm install --ignore-workspace
pnpm run --ignore-workspace

I get the local dependencies in by bundling them with esbuild:

#!/bin/bash
# File: bundle-node

if [[ -z $1 ]] || [[ -z $2 ]]; then
	echo "Usage: bundle-node <source-code-directory> <output-directory>"
	echo "The entry point in the source code directory is assumed to be index.ts"
	echo "Example: bundle-node src dist"
	exit 1
fi

esbuild $1/index.ts --bundle \
	--platform=node \
	--target=node20 \
	--outdir=$2 \
	--out-extension:.js=.cjs \
	--external:firebase-functions \
	--external:firebase-admin \
	--allow-overwrite \
	--sourcemap=external \
	--legal-comments=none

No copying, linking, zipping, vendoring or any of that nonsense. esbuild inlines my custom code right into the output file. The Google dependencies are left out, and Cloud Build takes care of installing them. So this generates a very small bundle.

The dummy package.json looks like this:

{
	"name": "@costvine/functions",
	"version": "0.0.1",
	"description": "Costvine Cloud Functions",
	"private": true,
	"engines": {
		"node": "20",
		"pnpm": ">=8"
	},
	"main": "index.cjs",
	"scripts": {
		"serve": "firebase emulators:start --only functions",
		"shell": "firebase functions:shell",
		"start": "firebase functions:shell",
		"deploy": "firebase deploy --only functions",
		"logs": "firebase functions:log"
	},
	"dependencies": {
		"@google-cloud/functions-framework": "^3.3.0",
		"firebase-admin": "^11.8.0",
		"firebase-functions": "^4.3.1"
	}
}

The firebase.json includes:

{
	"functions": [
		{
			"source": ".",
			"codebase": "jsdefault",
			"ignore": ["*.log", "node_modules", ".firebaserc", "firebase.json"]
		}
	],
	"emulators": {
		"functions": {
			"port": 5001
		},
		"ui": {
			"enabled": true
		},
		"singleProjectMode": true
	}
}

I can re-run bundle-node while the emulator is running and it will 'hot reload' my cloud function for testing.

[0x80]

@johndunderhill I tried it at some point in the beginning and somehow concluded that it wasn't doing what I needed. I think it doesn't take the original lockfile into account. It was quite a long and winding road for me to get to the isolated lockfile for pnpm, so I don't quite remember everything.

Eventually, the process of adapting the lockfile is quite straightforward, and also fast, so I don't think there's much to gain from using an alternative approach.

[solarpush]

I had the same problem and finally found a solution

In my case it's only types and therefore a devellopement dependency but the library is well built in the modules nodes so it should work for the rest.

I finally found the solution in the google doc...

Google docs local packages

Change this:
"@types/package" : "workspace:1.0.0"
To
"@types/package" : "file:1.0.0"
Before build and deployment and after
Reverse in postdeploy.

pre-deploy.js

const fs = require("fs");
fs.readFile("./package.json", "utf8", (err, data) => {
  if (err) {
    console.error("Erreur lors de la lecture du fichier package.json :", err);
    return;
  }
  const packageJson = JSON.parse(data);
  packageJson.devDependencies["@types/gcrm"] = "file:1.0.0";
  fs.writeFile(
    "./package.json",
    JSON.stringify(packageJson, null, 2),
    "utf8",
    (err) => {
      if (err) {
        console.error(
          "Erreur lors de l'écriture dans le fichier package.json :",
          err,
        );
        return;
      }
      console.log("Le fichier package.json a été mis à jour avec succès.");
    },
  );
});

Does the same but reverse process in post-deploy.js (or add conditionning for switch also)

firebase.json

"functions": [
    {
      "source": "apps/backend/functions",
      "codebase": "default",
      "ignore": [
        "node_modules",
        ".git",
        "firebase-debug.log",
        "firebase-debug.*.log"
      ],
      "predeploy": [
        "pnpm --prefix \"$RESOURCE_DIR\" run predeployscript",
        "pnpm --prefix \"$RESOURCE_DIR\" run lint",
        "pnpm --prefix \"$RESOURCE_DIR\" run build"
      ],
      "postdeploy": [
        "pnpm --prefix \"$RESOURCE_DIR\" run postdeployscript"
      ]
    }
  ],

Don't forget to create the script on your package.json

"scripts": {
    "lint": "eslint --ext .js,.ts .",
    "lint:fix": "eslint --ext .js,.ts --fix .",
    "format": "prettier --write \"src/**/*.ts\"",
    "build": "pnpm run format && tsc",
    "build:watch": "tsc --watch",
    "dev": "pnpm run build && firebase emulators:start --only functions",
    "shell": "pnpm run build && firebase functions:shell",
    "start": "pnpm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log",
    "test": "vitest",
    "test:ui": "vitest --ui",
    "predeployscript": "node ./pre-deploy.js",
    "postdeployscript": "node ./post-deploy.js"
  },

Bye