-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(serverless): Zip serverless dependencies for AWS Lambda
* Implement extension for AWS Lambda for automatic integration. * AWS Lambda packages cached The packages to upload to the AWS Lambda layer are zipped. This zip file is cached. * AWS Lambda build workflow Created another workflow to build the AWS Lambda layer by running a script when changes something on @sentry/serverless * Merged AWS workflow to the main one Removed the AWS Lambda workflow, and added the build to the main workflow. Also, included the generated zip file to the artifacts. * Serverless package versions updated * Deleted publish script The publish functionality should not be here, deleted. * Serverless AWS integration docs minor change Referenced the docs. Also, updated `yarn.lock`. * Set default value in the parameter Instead of setting a default value at the beginning of the function, so that the undefined is avoided, it's now set in the parameter. * Deleted dependency `path-exists` * Fixed broken promise Fixed the broken promise, and removed the `path-exists` dependency from `package.json`. * ref(aws-lambda): WIP build script * fix(aws-lambda): Correct API function Using the function available in the same node version. Also, removed some logs. * ref: Revert yarn.lock * ci: Update * ref: Reduce imports * ref: Remove dependency in AWS Lambda build script * ref: Rename AWS Lambda initializer Be more specific in the initializer script and rename it to `awslambda-auto`. * doc: Remove optional step Co-authored-by: Vladimir Kochnev <marshall-lee@evilmartians.com> Co-authored-by: Daniel Griesser <daniel.griesser.86@gmail.com>
- Loading branch information
3 people
committed
Dec 18, 2020
1 parent
66a6a8c
commit 36463ce
Showing
9 changed files
with
319 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| dist-awslambda-layer/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| const path = require('path'); | ||
| const process = require('process'); | ||
| const fs = require('fs'); | ||
| const childProcess = require('child_process'); | ||
|
|
||
| const findUp = require('find-up'); | ||
| const packList = require('npm-packlist'); | ||
| const readPkg = require('read-pkg'); | ||
|
|
||
| const serverlessPackage = require('../package.json'); | ||
|
|
||
| // AWS Lambda layer are being uploaded as zip archive, whose content is then being unpacked to the /opt | ||
| // directory in the lambda environment. | ||
| // | ||
| // So this script does the following: it builds a 'dist-awslambda-layer/nodejs/node_modules/@sentry/serverless' | ||
| // directory with a special index.js and with all necessary @sentry packages symlinked as node_modules. | ||
| // Then, this directory is compressed with zip. | ||
| // | ||
| // The tricky part about it is that one cannot just symlink the entire package directories into node_modules because | ||
| // all the src/ contents and other unnecessary files will end up in the zip archive. So, we need to symlink only | ||
| // individual files from package and it must be only those of them that are distributable. | ||
| // There exists a `npm-packlist` library for such purpose. So we need to traverse all the dependencies, | ||
| // execute `npm-packlist` on them and symlink the files into 'dist-awslambda-layer/.../@sentry/serverless/node_modules'. | ||
| // I didn't find any way to achieve this goal using standard command-line tools so I have to write this script. | ||
| // | ||
| // Another, and much simpler way to assemble such zip bundle is install all the dependencies from npm registry and | ||
| // just bundle the entire node_modules. | ||
| // It's easier and looks more stable but it's inconvenient if one wants build a zip bundle out of current source tree. | ||
| // | ||
| // And yet another way is to bundle everything with webpack into a single file. I tried and it seems to be error-prone | ||
| // so I think it's better to have a classic package directory with node_modules file structure. | ||
|
|
||
| /** Recursively traverse all the dependencies and collect all the info to the map */ | ||
| async function collectPackages(cwd, packages = {}) { | ||
| const packageJson = await readPkg({ cwd }); | ||
|
|
||
| packages[packageJson.name] = { cwd, packageJson }; | ||
|
|
||
| if (!packageJson.dependencies) { | ||
| return packages; | ||
| } | ||
|
|
||
| await Promise.all( | ||
| Object.keys(packageJson.dependencies).map(async dep => { | ||
| // We are interested only in 'external' dependencies which are strictly upper than current directory. | ||
| // Internal deps aka local node_modules folder of each package is handled differently. | ||
| const searchPath = path.resolve(cwd, '..'); | ||
| const depPath = fs.realpathSync( | ||
| await findUp(path.join('node_modules', dep), | ||
| { type: 'directory', cwd: searchPath }) | ||
| ); | ||
| if (packages[dep]) { | ||
| if (packages[dep].cwd != depPath) { | ||
| throw new Error(`${packageJson.name}'s dependenciy ${dep} maps to both ${packages[dep].cwd} and ${depPath}`); | ||
| } | ||
| return; | ||
| } | ||
| await collectPackages(depPath, packages); | ||
| }), | ||
| ); | ||
|
|
||
| return packages; | ||
| } | ||
|
|
||
| async function main() { | ||
| const workDir = path.resolve(__dirname, '..'); // packages/serverless directory | ||
| const packages = await collectPackages(workDir); | ||
|
|
||
| const dist = path.resolve(workDir, 'dist-awslambda-layer'); | ||
| const destRootRelative = 'nodejs/node_modules/@sentry/serverless'; | ||
| const destRoot = path.resolve(dist, destRootRelative); | ||
| const destModulesRoot = path.resolve(destRoot, 'node_modules'); | ||
|
|
||
| try { | ||
| // Setting `force: true` ignores exceptions when paths don't exist. | ||
| fs.rmSync(destRoot, { force: true, recursive: true, maxRetries: 1 }); | ||
| fs.mkdirSync(destRoot, { recursive: true }); | ||
| } catch (error) { | ||
| // Ignore errors. | ||
| } | ||
|
|
||
| await Promise.all( | ||
| Object.entries(packages).map(async ([name, pkg]) => { | ||
| const isRoot = name == serverlessPackage.name; | ||
| const destPath = isRoot ? destRoot : path.resolve(destModulesRoot, name); | ||
|
|
||
| // Scan over the distributable files of the module and symlink each of them. | ||
| const sourceFiles = await packList({ path: pkg.cwd }); | ||
| await Promise.all( | ||
| sourceFiles.map(async filename => { | ||
| const sourceFilename = path.resolve(pkg.cwd, filename); | ||
| const destFilename = path.resolve(destPath, filename); | ||
|
|
||
| try { | ||
| fs.mkdirSync(path.dirname(destFilename), { recursive: true }); | ||
| fs.symlinkSync(sourceFilename, destFilename); | ||
| } catch (error) { | ||
| // Ignore errors. | ||
| } | ||
| }), | ||
| ); | ||
|
|
||
| const sourceModulesRoot = path.resolve(pkg.cwd, 'node_modules'); | ||
| // `fs.constants.F_OK` indicates whether the file is visible to the current process, but it doesn't check | ||
| // its permissions. For more information, refer to https://nodejs.org/api/fs.html#fs_file_access_constants. | ||
| try { | ||
| fs.accessSync(path.resolve(sourceModulesRoot), fs.constants.F_OK); | ||
| } catch (error) { | ||
| return; | ||
| } | ||
|
|
||
| // Scan over local node_modules folder of the package and symlink its non-dev dependencies. | ||
| const sourceModules = fs.readdirSync(sourceModulesRoot); | ||
| await Promise.all( | ||
| sourceModules.map(async sourceModule => { | ||
| if (!pkg.packageJson.dependencies || !pkg.packageJson.dependencies[sourceModule]) { | ||
| return; | ||
| } | ||
|
|
||
| const sourceModulePath = path.resolve(sourceModulesRoot, sourceModule); | ||
| const destModulePath = path.resolve(destPath, 'node_modules', sourceModule); | ||
|
|
||
| try { | ||
| fs.mkdirSync(path.dirname(destModulePath), { recursive: true }); | ||
| fs.symlinkSync(sourceModulePath, destModulePath); | ||
| } catch (error) { | ||
| // Ignore errors. | ||
| } | ||
| }), | ||
| ); | ||
| }), | ||
| ); | ||
|
|
||
| const version = serverlessPackage.version; | ||
| const zipFilename = `sentry-node-serverless-${version}.zip`; | ||
|
|
||
| try { | ||
| fs.unlinkSync(path.resolve(dist, zipFilename)); | ||
| } catch (error) { | ||
| // If the ZIP file hasn't been previously created (e.g. running this script for the first time), | ||
| // `unlinkSync` will try to delete a non-existing file. This error is ignored. | ||
| } | ||
|
|
||
| try { | ||
| childProcess.execSync(`zip -r ${zipFilename} ${destRootRelative}`, { cwd: dist }); | ||
| } catch (error) { | ||
| // The child process timed out or had non-zero exit code. | ||
| // The error contains the entire result from `childProcess.spawnSync`. | ||
| console.log(error); // eslint-disable-line no-console | ||
| } | ||
| } | ||
|
|
||
| main().then( | ||
| () => { | ||
| process.exit(0); | ||
| }, | ||
| err => { | ||
| console.error(err); // eslint-disable-line no-console | ||
| process.exit(-1); | ||
| }, | ||
| ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| import * as Sentry from './index'; | ||
|
|
||
| const lambdaTaskRoot = process.env.LAMBDA_TASK_ROOT; | ||
| if (lambdaTaskRoot) { | ||
| const handlerString = process.env._HANDLER; | ||
| if (!handlerString) { | ||
| throw Error(`LAMBDA_TASK_ROOT is non-empty(${lambdaTaskRoot}) but _HANDLER is not set`); | ||
| } | ||
| Sentry.AWSLambda.init(); | ||
| Sentry.AWSLambda.tryPatchHandler(lambdaTaskRoot, handlerString); | ||
| } else { | ||
| throw Error('LAMBDA_TASK_ROOT environment variable is not set'); | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.