ESM support stopped working in Cloud Functions (GCF)

[koistya]

ESM support stopped working today in Google Cloud Functions (GCF) environment with Node.js 16 runtime. Trying to deploy an ESM codebase fails with the error:

To load an ES module, set "type": "module" in the package.

package.json

{
  "name": "esm",
  "type": "module",
  "exports": "./index.js",
  "packageManager": "yarn@3.2.0-rc.12"
}

index.js

export function esm(req, res) {
  res.send("PASS");
}

$ cloud functions deploy esm \
    --project=example --region=us-central1 --allow-unauthenticated --trigger-http \
    --entry-point=esm --memory=256MB --runtime=nodejs16 --source=. \
    --set-env-vars="NODE_OPTIONS=--loader=./.pnp.loader.mjs --require=./.pnp.cjs --no-warnings"

Workaround (if you want to preserve ESM in your project)

1. Bundle the code. E.g. by using Rollup and Babel wtih @babel-preset/env, target: { node: "14" } preset
2. Deploy to Node.js v14 GCF runtime. See example: GraphQL API and Relay Starter Kit → api/rollup.config.js

Ref #233, #292, nodejs/node#41189, nodejs/node#41198

[rileytomasek]

I'm also seeing this but noticed that switching back to Node 14 deploys successfully...

[chiahung-keng]

Same here.. ESM support is not working with Node.js 16 runtime.. Any update from Google?

[matthewrobertson]

Sorry to hear you are having trouble with this feature. I just tried this out for myself and I wasn't able to repro the issue. Could you please share more information about your project structure?

[matthewrobertson]

I see what is going on. The failure happens when building an ESM function with our GCP Buildpacks. We run node check on the entrypoint file here. It seems like that is failing in nodejs16 regardless of type setting in the package.json or the esm file extension:

node --check foo.mjs
(node:2290922) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/usr/local/google/home/mattrobertson/gcf/functions-framework-nodejs/test/data/esm_mjs/foo.mjs:12
export {testFunction};
^^^^^^

SyntaxError: Unexpected token 'export'
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1031:15)
    at checkSyntax (node:internal/main/check_syntax:66:3)
    at node:internal/main/check_syntax:39:3

[matthewrobertson]

This appears to be a regression in v16 of the Node.js engine. I have filed nodejs/node#41189. Unfortunately, the work around is to use nodejs14 on GCF for now.

[chiahung-keng]

Thanks for the update Matthew.

[jthoward64]

Well there went a few hours of tinkering with my package.json; so the fix is basically just wait for a node update? Or is is this something GCP is working on? In the meantime node 14 it is!

[taeold]

@matthewrobertson Is it possible to revert the version of node v16 used in the GCF runtime?

[jthoward64]

@matthewrobertson Is it possible to revert the version of node v16 used in the GCF runtime?

That seems like best stopgap given that the update was so recent

[webervin]

Actually even better solution would be to allow buildpacks take in and preserve NODE_OPTIONS from environment, that way ones who need it can pass NODE_OPTIONS=--experimental-modules which helps currently used node --check to understand and allow modules. This is both feature request (to allow custom NODE_OPTIONS build) and solution to this issue which is not blocked by nodejs release, testing etc times.

[jthoward64]

Actually even better solution would be to allow buildpacks take in and preserve NODE_OPTIONS from environment, that way ones who need it can pass NODE_OPTIONS=--experimental-modules which helps currently used node --check to understand and allow modules. This is both feature request (to allow custom NODE_OPTIONS build) and solution to this issue which is not blocked by nodejs release, testing etc times.

@webervin Maybe open a new issue referencing this comment and this issue? (the ... button on the top right of your comment)

[matthewrobertson]

I think the easiest fix here is to just skip the node --check in the functions framework buildpack when the app is using ESM. I am happy to make that change; however, we are not scheduled to push a new release of the buildpacks until the new year.

[matthewrobertson]

@webervin it is already possible to set build time environment variables when deploying functions, but I am not sure setting NODE_OPTIONS=--experimental-modules will help here.

[jthoward64]

A fix was pushed in node about 9 days ago, it's just a waiting game for node to release the change. nodejs/node#41189 , nodejs/node#41198 , and nodejs/node@3f7dabd

[50an6xy06r6n]

@matthewrobertson is there an ETA on when this will be fixed?

[matthewrobertson]

I was really hoping that Node.js would have released a fix for this by now, but I just tested with v16.13.2 (npm v8.1.2) and it is still an issue. I will add a temporary workaround in the buildpacks.

[matthewrobertson]

side note: it does seem to be fixed in nodejs v17.4.0 (npm v8.3.1). So perhaps the fix needs to be manually back ported to 16. Would be awesome if someone could ask them to do that.

[taeold]

The needed commit is already scheduled for backporting as indicated by the lts-watch label in nodejs/node#41198.

We'll have to wait until the 16 LTS backporter is ready to backport and cut a new release...

[jthoward64]

I'd say go for a temporary workaround until whenever Node gets around to a new release of 16 (anyone know when that might be?)

[algoad]

Is there any plans on resolving this soon? I'm using another package that requires Node 16 so going back to node 14 engine is not an option for me.

[grant]

Is there any plans on resolving this soon? I'm using another package that requires Node 16 so going back to node 14 engine is not an option for me.

Understood. Matt is actively working on a solution that is visible internally #407 (comment). There's no update since yesterday.

[50an6xy06r6n]

When can we expect these changes to be deployed to prod?

[matthewrobertson]

The work around is rolling out now. It should be in all GCP regions by the end of the week.

[DibyodyutiMondal]

Just to be clear: this will not be needing a rollback to node 14, right?

[alexanderniebuhr]

@matthewrobertson has the fix landed in all GCP regions yet? I am still seeing this issue in the europe-west1 region

[christianp86]

@matthewrobertson has the fix landed in all GCP regions yet? I am still seeing this issue in the europe-west1 region

Yeah same in europe-west3

[matthewrobertson]

The rollout completed a few hours ago. I just confirmed that I can deploy an ESM function in europe-west1 with the nodejs16 runtime. Please re-open the bug if you are still having issues.

[koistya]

Might be not related to this particular update, but coincidently Yarn-based cloud functions stopped working:

ERROR: (gcloud.alpha.functions.deploy) OperationError:
  code=3, message=Build failed with status:
    FAILURE and message: Unknown Syntax Error: Invalid option name ("--production=false").

https://github.com/yarnpkg/berry/blob/43ccde3f8bfb5fd825d272dc2e305f8e4f4171dc/packages/plugin-essentials/sources/commands/install.ts#L102

Ref GoogleCloudPlatform/buildpacks#174