Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Custom Resource functions to use Node.js 18 #12133

Open
1 task done
michalf-kainos opened this issue Aug 30, 2023 · 26 comments · May be fixed by #12146 or #12445
Open
1 task done

Upgrade Custom Resource functions to use Node.js 18 #12133

michalf-kainos opened this issue Aug 30, 2023 · 26 comments · May be fixed by #12146 or #12445

Comments

@michalf-kainos
Copy link

Is there an existing issue for this?

  • I have searched existing issues, it hasn't been reported yet

Use case description

Node.js 16 is going out of support (security support ends on 11 Sep 2023).

The Lambda function for custom resources have nodejs16 runtime hardcoded:

const customResourceFunction = {
Type: 'AWS::Lambda::Function',
Properties: {
Code: {
S3Bucket,
S3Key,
},
FunctionName: absoluteFunctionName,
Handler,
MemorySize: 1024,
Runtime: 'nodejs16.x',
Timeout: 180,
},
DependsOn: [],
};

In multiple projects Lambda functions using unsupported nodejs version would be highlighted as security risk.

Proposed solution (optional)

Update customResourceFunction.runtime to nodejs18.x

@svoychik svoychik linked a pull request Sep 13, 2023 that will close this issue
@doblinger-extron
Copy link

Relates to #12116

@ptitkosmos
Copy link

I suggest to retrieve the runtime provide by the user in the serverless template :

provider:
  runtime: nodejs18.x

@mikeprimm
Copy link

mikeprimm commented Dec 20, 2023

Just got notices from AWS on end-of-support for node 16 runtime - end of standard support is 11 Mar 2024 (deprecated support ending 11 Apr 2024). I'd suggest move fixed value to nodejs 20.x (or use the runtime from the provider->runtime setting, as suggested by @ptitkosmos ).

EDIT: notice has different date than the lambda runtime page, but still coming relatlively soon - and node16 has been out of support since September 2023.

"As described in the Lambda runtime support policy [2], end of support for language runtimes in Lambda happens in several stages. Starting on June 12, 2024, Lambda will no longer apply security patches and other updates to the Node.js 16 runtime used by Lambda functions, and functions using Node.js 16 will no longer be eligible for technical support. Also, Node.js 16 will no longer be available in the Console, although you can still create and update functions using Node.js 16 via AWS CloudFormation, the AWS CLI, AWS SAM, or other tools. Starting July 15, 2024, you will no longer be able to create new Lambda functions using the Node.js 16 runtime. Starting August 15, 2024, you will no longer be able to update existing functions using the Node.js 16 runtime."

@janik6n
Copy link

janik6n commented Dec 22, 2023

Also relates to issue #12265

@gavinjohnson-crowdcube
Copy link

"Starting August 15, 2024, you will no longer be able to update existing functions using the Node.js 16 runtime." is the killer. So in 8 months in cases where the custom resources are used, unless something is changed then serverless framework will simply fail. You'll no longer be able to deploy. 😨

Seriously thinking it's time to migrate away. 🤔

@NickMainAtPI
Copy link

Is there any hope of a fix for this issue being merged? Or should my team spend our time before August 15th migrating our lambda infrastructure to another solution?...

@Mmarzex
Copy link
Contributor

Mmarzex commented Feb 6, 2024

Hi everyone, sorry we've been slow to respond here. This change will be available in the Serverless v4 beta that will be releasing within the next week.

@sarangpatel
Copy link

sarangpatel commented Feb 7, 2024

@Mmarzex I hope next week. For time being any hacky solution please provide? so that we cannot sit idle till next week ;)

@noumanbhatti
Copy link

does somebody found any hack to this ?

@ofek-kenbi
Copy link

ofek-kenbi commented Feb 22, 2024

@noumanbhatti @sarangpatel
I have a very ugly hack but at least it's working.
Patch the "node_modules/serverless/lib/plugins/aws/custom-resources/generate-zip.js" file using something like patch-package.
The edited file:

'use strict';

/* This files patches serverless generate-zip.js to include aws-sdk in the custom-resources lambda
   This is to resolve issue where aws-sdk is not included in node.js runtime >= 18.x
   This is a temporary fix until serverless framework fix this issue:
   https://github.com/serverless/serverless/issues/12133 */
const path = require('path');
const fse = require('fs-extra');
const getTmpDirPath = require('../../../utils/fs/get-tmp-dir-path');
const createZipFile = require('../../../utils/fs/create-zip-file');
const ensureArtifact = require('../../../utils/ensure-artifact');
const safeMoveFile = require('../../../utils/fs/safe-move-file');
const childProcess = require('child_process');
const util = require('util');

const exec = util.promisify(childProcess.exec);

const srcDirPath = path.join(__dirname, 'resources');

const artifactName = 'custom-resources-patched.zip';

module.exports = async () => {
  const resultPath = await ensureArtifact(artifactName, async (cachePath) => {
    const tmpDirPath = getTmpDirPath();
    const tmpInstalledLambdaPath = path.resolve(tmpDirPath, 'resource-lambda');
    const tmpZipFilePath = path.resolve(tmpDirPath, 'resource-lambda.zip');
    const cachedZipFilePath = path.resolve(cachePath, artifactName);
    await fse.copy(srcDirPath, tmpInstalledLambdaPath);
    console.log("Patching custom-resource lambda....");
    await fse.writeJSON(path.join(tmpInstalledLambdaPath, 'package.json'), { name: "custom-resources", dependencies: { "aws-sdk": "2.1553.0" } });
    await exec('npm install', { cwd: tmpInstalledLambdaPath });
    await createZipFile(tmpInstalledLambdaPath, tmpZipFilePath);
    await safeMoveFile(tmpZipFilePath, cachedZipFilePath);
  });
  return path.resolve(resultPath, artifactName);
};

You will also need to patch the serverless/lib/plugins/aws/custom-resources/index.js file and update the version to 18/20.

@gupta-abhi26
Copy link

Hey @Mmarzex, any updates on the release ?

@sarangpatel
Copy link

@Mmarzex we are getting tense, because of this. plz update update

@smiley717
Copy link

Hi everyone, sorry we've been slow to respond here. This change will be available in the Serverless v4 beta that will be releasing within the next week.

When is the ETA of this? June is not so far away from now.

@bitsofinfo
Copy link

This won't be backported into 3.x? Concerned having to force move to a new major (especially a beta SLS) version will bring more risk rather than just having SLS 3.x patched for this change...

@gavinjohnson-crowdcube
Copy link

v4 alpha looks to be out now https://github.com/serverless/serverless/tree/v4.0

@gavinjohnson-crowdcube
Copy link

Looks like the custom resource upgrade to node20.x is in it as well bd8cef3

@MatthiasBoehm87
Copy link

Is there any chance that this will be backported into 3.x? There isn't much time left until AWS will deny a lambda update completely, and then we won't be able to deploy our projects anymore.

@NickMainAtPI
Copy link

Any updates on this, especially considering the lack of full V4 rollout at this point?

@metafounder
Copy link

I am also interested in updates on this.

@pd-alex
Copy link

pd-alex commented Apr 22, 2024

It would be great to hear an update on this.
The AWS cut-off date is approaching.

@wesyoung
Copy link

wesyoung commented Apr 22, 2024

this seems to work:

  • add - lambda-update-deprecated-runtime to your plugin section in serverless.yml
  • mkdir .serverless_plugins add the js below to .serverless_plugins/lambda-update-deprecated-runtime.js
  • run sls package and vi .serverless/cloudformation-template-update-stack.json to verify you see node20 in the right place:
"Handler": "api-gateway-cloud-watch-role/handler.handler",
        "MemorySize": 1024,
        "Runtime": "nodejs20.x",
        "Timeout": 180,
       ...

ref: https://serverlesscode.com/post/customizing-serverless-with-plugins/

(i rlz the general guidance right now is a little wonky, hth... ty for those that have supplied examples thus far!)

lambda-update-deprecated-runtime.js

'use strict';

class LambdaUpdateDeprecatedRuntime {

  constructor(serverless, options) {
    this.serverless = serverless;
    this.options = options;
    this.provider = 'aws';

    this.hooks = {
      'before:package:finalize': this.afterCompileEvents.bind(this),
    };
  }
  afterCompileEvents() {
    let key = 'CustomDashresourceDashapigwDashcwDashroleLambdaFunction'
    let resources = this.serverless.service.provider.compiledCloudFormationTemplate.Resources;
    if (key in resources && resources[key].Properties.Runtime == 'nodejs16.x') {
      this.serverless.cli.log("Fixed CustomDashresourceDashexistingDashs3LambdaFunction runtime from `nodejs16.x` to `nodejs20.x`");
      resources[key].Properties.Runtime = 'nodejs20.x'
    }
  }
}

module.exports = LambdaUpdateDeprecatedRuntime;

@jonasduever
Copy link

Thanks @wesyoung! But that would mean that this is not true? (Would surprise me to be honest.)

However we can't just increment the runtime version. These custom resource handlers use aws-sdk v2 which is not bundled with node18 lambda functions. In order to change this, you'd need to update the custom resource handlers to use v3 of the AWS SDK.

@wesyoung
Copy link

Thanks @wesyoung! But that would mean that this is not true? (Would surprise me to be honest.)

However we can't just increment the runtime version. These custom resource handlers use aws-sdk v2 which is not bundled with node18 lambda functions. In order to change this, you'd need to update the custom resource handlers to use v3 of the AWS SDK.

maybe- it makes the error (notice) go away for stacks that are already created. i'll guess i'll have to destroy and rebuild a stack to test (?)

@wesyoung
Copy link

Thanks @wesyoung! But that would mean that this is not true? (Would surprise me to be honest.)

However we can't just increment the runtime version. These custom resource handlers use aws-sdk v2 which is not bundled with node18 lambda functions. In order to change this, you'd need to update the custom resource handlers to use v3 of the AWS SDK.

maybe- it makes the error (notice) go away for stacks that are already created. i'll guess i'll have to destroy and rebuild a stack to test (?)

@jonasduever i think you're right, you can deploy the change to get rid of the notice, but if you try to delete and then re-create that part of the stack (the logs section), CF hangs for a good 30-40min and then fails, which highlights your point i think. guess we're stuck...

(gofundme donation to serverless ? it's hard running a biz on opensource sometimes... :))

@xoapit
Copy link

xoapit commented Apr 23, 2024

It should be fixed by #12146 but the maintainer did not review and merge it :(

@bradwood
Copy link

🥱

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet