New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Serverless-Webpack watching for changes not working #931
Comments
Same here, only that I cannot downgrade because I use Lambda.invoke implemented on v6. Try setting the config bellow, it will work because for each time it calls the process it reloads the files from file system, when using InProcessRunner it caches the import value from handlerPath and keeps it in memory
|
Can you try a tool like nodemon? |
@dherault nodemon does not work because.. even a very small application takes 11 seconds to start. This makes serverless-offline useless for me :( Please readd hot loading this breaks serverless development for me and many others - also not having nodemon was one of the features in the old readme... |
I'll have to check my setup, but serverless-webpack is rebuilding for changes with me |
Also finding the serverless-offline@6.1.0 is breaking hot reload. Reverting back to 5.12.0 makes hot reload work again. Here is my other related plugin versions:
|
We implemented this as a stop gap until hot reloading is implemented properly: It seems to work ok for us using serverless-bundle, theoretically it should work for serverless-webpack as well. Instructions:
plugins:
- serverless-offline
- ./plugins/offline-invalidate |
@cchamplin now that you have mastered the plugin, could you offer this technical solution as a PR and an option? |
That worked for me thanks! |
Unfortunately, this doesn't seem to work with web sockets when running with serverless-offline@6.1.4 and serverless-webpack@5.3.1. It is catching the changes for http handlers, but not web socket handlers. Not related to your workaround, but in general, it seems like hot reloading isn't working for web socket handlers in serverless-offline@5.12.1 or 5.12.0 either. |
@cchamplin @dherault Would love to see the hot reloading workaround go into this repo, please let us know when it has been merged and we will make the upgrade to the latest version. |
I'm also experiencing this issue. Following this and looking forward to a patch release. |
Still waiting for the fix, I don't understand why it' taking so much time to be solved, it's a major regression... |
We are using npm-watch. We have our lambdas separated by service in folders so watch config looks like this in package.json "scripts": {
"dev": "npm-watch dev:serverless",
"dev:serverless": "sls offline start"
},
"watch": {
"dev:serverless": {
"patterns": [
"*Lambda/**"
],
"extensions": "js",
"quiet": true
}
},
"devDependencies": {
"npm-watch": "^0.6.0",
"serverless-dotenv-plugin": "^2.4.2",
"serverless-offline": "^6.4.0"
} |
Yes I confirm this works. It also has an advantage that you can specify the directory or files to watch. Because I was using globbing to add files dynamically at runtime, Webpack was not aware of file changes. By the way, Webpack has ignore settings but you can not specify what to watch. |
Waiting for a fix... |
Any update here? |
Added a fix that will remove the module from the "require" cache, before retrying the handler. The drawback is any "global" variables would be reset as well, but I see that as a plus as we shouldn't be using global variables. If you want caching turned on add the following to your configuration.
or
I set it to default to not cache, as this is generally a development tool. |
useChildProcess messes up the source-maps |
Has this PR been merged yet? |
the issue still shows the PR in an open state |
The solution I found is,
and for the source-map issue comes with useChildProcesses (#1039), using 'https://www.npmjs.com/package/source-map-support' |
(Task manager without
(Task manager with Every time I invoke my API, a childProcessHelper is created and the debugger is attached. So after sometime debugger freezes (sls-offline server freezes). I think this is the issue authors are trying to solve. Hot reloading without memory leaks. Without debugger, I ran API calls until memory is fully used. But the server won't get freeze. Some node tasks are killed or paused by the OS to make room for the new child processes. But in debugging, the debugger gets exhausted by getting attached to every child process and freezes. @dherault any comment on this? |
I have the same issue with child processes which is why i made a fix for non child processes. I have a project where after about an hour I have over 2000 child processes. Ctrl-C-ing the serverless window takes 30-40 seconds to close, and while its running, other processes lock up if i don't renice before i start. Before the fix, I would turn child processes off when not working serverless (the backend), and when i was actively changing code, i'd turn it on, so hot loading would work. Without child processes, i've run the server for days without needing a restart. |
we place exit() just before returning the results it autokills your process when you use useChildProcesses not ideal but fine for us :) (we have one main function) |
I too use async handlers. But it won't kill created child processes. |
Issue #931 fix for not allowing caching
Fix was merged, and released as v6.7.0, update and see if it works for you [EDIT] Verified that it was working within my projects as well |
@dl748 it partially solves the issue since this fix can't be used with persistent variables (used by AWS modules and required for persistent connections cf https://docs.atlas.mongodb.com/best-practices-connecting-to-aws-lambda/ and other performance optimisations).
|
Just wanted to share my feedback as I've also been trying to find the best workaround to this issue. Unfortunately #1050 has not been a viable solution for me. The function I'm testing with a fairly standard graphql handler implemented with Configurations Testedv6.7.0 - Default SettingsResponse: 502 Bad Gateway
v6.7.0 -
|
I took a shot at an alternative solution in #1090. Someone mentioned in another thread that setting the idle timeout low (say 5s) was a viable workaround. This gave me the idea to simply trigger the lambda cleanup in response to the This does require you to use worker threads, but I haven't seen any downsides to doing so thus far. Definitely still a work in progress, but please share your feedback if you give it a try. |
No
Nice idea, unfortunately, the problem is bigger than just webpack, and i don't think adding specific code code for supporting a specific plugin is the correct solution. |
The only thing specific to I also tend to agree with @jer-sen that reloading should be triggered only by an actual change to the code. The current solution with |
Since there is no universal method for serverless plugins to notify other plugins of content changes, this still breaks non serverless-webpack for hot reloading. A solution for only one plugin is not really a solution at all, in fact, this "code" would be better in serverless-webpack, and not in serverless-offline. |
@dl748 While it would be nice, is a universal method supported by I've created a POC (#1093) that exposes the ability to trigger the lambda cleanup as an entrypoint command. While this won't (currently) work out of the box, support could certainly be added in other plugins like |
I tried the hook way first, I didn't like it due to the fact that its a lot of work for a plugin writer to do, to find the hook and then call it. Its closely linked to commands and is not suppose to have arguments. So I opted to go a different way so not to break existing functionality. So I have a proof of concept that works, but its not "standard" for serverless plugins. So what I did was hijack the PluginManager and add a new variable .events in the ServerlessOffline constructor, and bound the contentChanges event to a function in class (similar to hooks)
then I created a standard async function to handle it
Now that its all set up... in the serverless-webpack https://github.com/serverless-heaven/serverless-webpack/blob/ebe825ee24890ae3bf9c896258daa715528e1e74/lib/wpwatch.js#L113
Now any plugin that makes changes to files just signals the contentChanges event on the PluginManager and passes an object with { files: [] } array. This is just a work in progress/concept, and I welcome any discussion. There is A LOT of work that can be done here though, but i'd like it to be as extensible as possible. I'd like to update PluginManager to support this but getting any PR's accepted into serverless is like pulling teeth. The nice thing about this is that now ANY other plugin can register events (custom) and provide a layer of inter plugin communication. That would at least run my clearModule function only on serverless-webpack compiles |
I believe it should. A interplugin communication layer should be created in the PluginManager for ALL plugins. |
Back to 5.12.0 it is then. |
Linking to PR #1093 in case it fixes this hot-reload issue, in combination with the newest version |
this create dozens of node instances and cause slowness of the whole machine - when trying to debug vs code in mac os - so not sure its a good idea to try it when working with VS code in mac os |
Wow, March 2020, this is... wow |
Combining the two serverless-offline options (allowCahce, useChildProcesses) works for me. It allows hot reloading and prevents memory leaks from accumulating across runs. I'm not sure how it performs if there are many simultaneous requests going though.
|
I've been having this issue ever since upgrading my
|
I think running server with
|
Instead using
|
adding --reloadHandler works but it honestly makes the process run very slow. |
works like charm , thanks
|
this worked for me. |
Bug Report
Current Behavior
If you change the code, webpack reloads but the changes are not reflected without restarting the process.
Expected behavior/code
Changing the code should reflect the changes without having to restart the server. Downgrading to serverless-offline to 5.12.1 works.
Environment
serverless
version: 1.67.0serverless-offline
version: 6.0.0node.js
version: v12.7.0OS
: macOS 10.15.3The text was updated successfully, but these errors were encountered: