Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"serverless-offline" v6 alpha feedback #768

Closed
55 of 67 tasks
dnalborczyk opened this issue Aug 5, 2019 · 139 comments
Closed
55 of 67 tasks

"serverless-offline" v6 alpha feedback #768

dnalborczyk opened this issue Aug 5, 2019 · 139 comments
Assignees

Comments

@dnalborczyk
Copy link
Collaborator

dnalborczyk commented Aug 5, 2019

Thank you for giving serverless-offline v6 alpha a try! 🎉

In this issue thread you can provide feedback regarding ongoing work for v6.

The current v6 releases are mainly marked as an alpha version to signal that stuff might be breaking, due to major code refactoring, reasons listed below (breaking changes), as well as the current lack of tests (which hopefully will improve over time). If you encounter any bugs please open a new issue stating the alpha version you are using.

Installation

npm i serverless-offline@next --save-dev

Latest release

v6.0.0 alpha.66

New features

  • (initial) OpenAPI/swagger support [alpha.10]
  • Worker thread support (requires Node.js v11.7.0+) [alpha.13]
  • Lambda Function Pool to better simulate Lambda behavior and keep Lambdas alive [alpha.16]
  • Python and Ruby users should be able to use websocket events now [alpha.22]
  • add schedule event source [alpha.47]
  • (initial) docker support [alpha.54] (thanks to @frozenbonito)
  • display memory usage

Planned new features

Maybe new features

Improvements

  • Refactoring Python handler execution Runs very slowly with AWS Python Lambda functions #742 [alpha.15]
  • Refactoring Ruby handler execution [alpha.17]
  • Added AWS Lambda environment variables [alpha.15]
  • Billed Duration was added to the console log [alpha.23]
  • use high resolution timer for duration and billed duration [alpha.27]
  • ES6 modules [alpha.28]
  • rollup build (chunked) [alpha.28] subsequently removed in [alpha.42]

Breaking changes (see below for a migration path)

  • default hot-reloading of handler module was removed [alpha.32]
  • remove --binPath option [alpha.6]
  • remove --exec option [alpha.9]
  • remove --location option [alpha.4]
  • remove --noEnvironment option [alpha.9]
  • remove --showDuration option, lambda execution time will be displayed by default [alpha.1]
  • remove --providedRuntime option [alpha.8]
  • remove --region option [alpha.14]
  • remove --stage option [alpha.14]
  • remove --prefix option [alpha.29]
  • remove --preserveTrailingSlash option [alpha.29]
  • useSeparateProcesses was renamed to useChildProcesses [alpha.32]
  • remove --skipCacheInvalidation [alpha.32]
  • remove --cacheInvalidationRegex [alpha.32]
  • remove event.isOffline=true [alpha.9]
  • remove process.env.IS_OFFLINE=true [alpha.25]
  • remove functionality shared with https://github.com/svdgraaf/serverless-plugin-stage-variables [`alpha.14]

Temporary Breaking changes

  • timeout feature has been removed temporarily to ease refactoring, (will be reimplemented soon) [alpha.0]

Bugfixes

Performance

  • ipcProcess loads now on first request as well as the module containing the handler [alpha.4]
  • only load and instantiate Api Gateway and Api Gateway WebSocket if used [alpha.8]
  • lazy load process runners [alpha.12]
  • lazy load in-process handlers [alpha.12]

Other

  • remove experimental warning for WebSocket support [alpha.9]
  • add apollo-server-lambda (graphql) scenario test
  • fix failing Python 2 tests [alpha.6]
  • fix failing Ruby tests [alpha.6]
  • add tests for Python3 support
  • add tests for Go support
  • add tests for Java 8 support
  • add tests for Dotnet Core support
  • fix failing (skipped) python tests on windows (not a blocker)

Migration Paths

--binPath option has been removed

we're not invoking serverless invoke local anymore, instead we moved the (lightly modified) code into the plugin and calling it ourselves. this is also way faster: ~ 25 times (on a mac, with python)

--exec option has been removed

just call your script as part of a shell script, npm script, javascript etc.

why was this option removed?

  • feature creep
  • maintenance burden
  • no script hook for server shut down (only startup)
  • possible issues with supporting multiple OS (linux, macos, windows)

--location option has been removed

if your handler file has different locations depending on the environment.

instead, use custom variables!

e.g.

custom:
  location:
    development: some-other-path/src
    production: src

function:
  hello:
    handler: ${self:custom.location.${opt:stage}}.hello
    # or alternatively, whichever you prefer
    # handler: ${self:custom.location.${env:NODE_ENV}}.hello

why was this option removed?

  • feature creep
  • functionality can easily be achieved by using custom variables
  • parity with serverless. serverless doesn't know about --location and therefore local invokes (and possibly other things) would be broken
  • serverless might add this (or a similar) functionality to the main framework, which would complicate things even more

--noEnvironment option has been removed

  • feature creep
  • there shouldn't be a need for such a functionality, as everything should be manageable through custom variables
  • If you used this feature and you need help migrating, please file an issue

why was this option removed?

  • parity with serverless, as serverless invoke local should also cause problems

--showDuration option has been removed

nothing needs to be done, it just works

why was this option removed?

  • because it would be redundant. :)

--providedRuntime option has been removed

instead, use custom variables!

at provider level, e.g.:

custom:
  runtime:
    development: nodejs10.x # use supported runtime here
    production: provided

provider:
  runtime: ${self:custom.runtime.${opt:stage}}
  # or alternatively, whichever you prefer
  # runtime: ${self:custom.runtime.${env:NODE_ENV}}

or at function level, e.g.:

custom:
  runtime:
    development: nodejs10.x # use supported runtime here
    production: provided

function:
  hello:
    handler: handler.hello
    runtime: ${self:custom.runtime.${opt:stage}} # or alternatively ${env:NODE_ENV}
    # or alternatively, whichever you prefer
    # runtime: ${self:custom.runtime.${env:NODE_ENV}}

why was this option removed?

--region option has been removed

coming soon

--stage option has been removed

coming soon

--prefix option has been removed

coming soon

--preserverTrailingSlash option has been removed

coming soon

event.isOffline=true has been removed

  • just check process.env.YOUR_OWN_VARIABLE instead (see below)

process.env.IS_OFFLINE=true has been removed

  • you can just provide your own environment variable, in e.g. serverless.yml:
# in the provider definition:
provider:
  environment:
    IS_OFFLINE: true

# or in the function definition
functions:
  hello:
    handler: handler.hello
    environment:
      IS_OFFLINE: true

why was this option removed?

  • for parity with the AWS event object

remove functionality shared with https://github.com/svdgraaf/serverless-plugin-stage-variables

coming soon

This list will be updated with each alpha release.

@computerpunc
Copy link
Contributor

computerpunc commented Aug 6, 2019

@dnalborczyk
What would be the recommended way of pushing updates to v6 alpha?

I’m thinking of:

  1. Fixing Fatal error = status code 200? #355 and callback with error responds with 200 OK when in AWS the response is 502 #678.
  2. Moving all current WebSocket tests Into integration phase so everyone (including CI) can run WebSocket tests and not just me :)

@dnalborczyk
Copy link
Collaborator Author

dnalborczyk commented Aug 6, 2019

@computerpunc awesome!

@dnalborczyk
What would be the recommended way of pushing updates to v6 alpha?

just do a PR against master. I didn't branch off, as it originally wasn't planned to be what it became.
there's a 5.x-branch for v5 releases, but I'd rather do that only if absolutely necessary, as the code base diverged too much.

I’m thinking of:

  1. Fixing Fatal error = status code 200? #355 and callback with error responds with 200 OK when in AWS the response is 502 #678.

sure, go for it! it would be great if you could also add some tests, similar to: https://github.com/dherault/serverless-offline/tree/master/src/__tests__/integration/handler

I also want to move everything non-unit test out of the "__tests__" folders, as those should be only used for unit tests.

I'm also planning an additional (optional) test script to automatically deploy to AWS and run the same tests against the deployment and expect the same results. that would be most likely done only locally by any developers or users, unless we have or get an AWS test account.

regarding #678 I introduced a LambdaFunction class, which also runs the lambda handler. unit tests are included. (unit tests for exceptions I have in a stash, not in master yet). we probably eventually need some generic version of how to reply to 'function errors' with 'lambda', 'proxy', 'websocket' etc.

  1. Moving all current WebSocket tests Into integration phase so everyone (including CI) can run WebSocket tests and not just me :)

that would be great!

btw, I wanna look at the websocket-authorizer merge later today so we can merge into master soon. maybe I'll move the test folders afterwards, otherwise it causes more friction.

@computerpunc
Copy link
Contributor

@dnalborczyk

Python integration test fails.

python tests › should work with python

    FetchError: invalid json response body at http://localhost:3000/hello reason: Unexpected end of JSON input

@dnalborczyk
Copy link
Collaborator Author

dnalborczyk commented Aug 10, 2019

hey @computerpunc

thank you! just fixed it. seems like it was only failing on windows because shebangs are not supported, and it tried to run serverless with the windows scripting host and not node.

@computerpunc
Copy link
Contributor

@dnalborczyk

I'm running on OSX. Now both Ruby and Python fail. This is latest plain vanilla master.

python tests › should work with python

    FetchError: invalid json response body at http://localhost:3000/hello reason: Unexpected end of JSON input

ruby tests › should work with ruby

    FetchError: invalid json response body at http://localhost:3000/hello reason: Unexpected end of JSON input

@dnalborczyk
Copy link
Collaborator Author

dnalborczyk commented Aug 10, 2019

@computerpunc

you probably have to run npm ci locally. I added a new dependency: execa, to fix spawning a process on windows. let me know if that fixes it.

@computerpunc
Copy link
Contributor

@dnalborczyk

you probably have to run npm ci locally.

It didn't fully help. Python still fails. Below is the errors generated:

Serverless: GET /hello (λ: hello)
Proxy Handler could not detect JSON:

Proxy Handler could not detect JSON:   Error --------------------------------------------------

  spawn python2 ENOENT

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.7.0
     Serverless Version:        1.49.0
     Enterprise Plugin Version: 1.3.8
     Platform SDK Version:      2.1.0

@dnalborczyk
Copy link
Collaborator Author

I'm running on OSX.

me too. plus the tests on travis are passing: https://travis-ci.org/dherault/serverless-offline/builds/570274311

I believe ruby and python are pre-installed on macOS.

I'm not a python nor a ruby user, but we should probably also check if python and ruby are installed on the machine where the tests are to be run, and if not, skip them. I'm gonna look into this.

could you run: python --version and ruby --version on your box just to make sure?

also, could you run the following:
go to: __tests__/integration/python

and then run: npx serverless invoke local -f hello

@computerpunc
Copy link
Contributor

@dnalborczyk

Ruby test runs OK. Python doesn't but I checked on another Mac it runs OK.

could you run: python --version and ruby --version on your box just to make sure?

On the problematic Mac Python 2.7.10 and on the OK Mac Python 2.7.16.

and then run: npx serverless invoke local -f hello

Same error as before (on the problematic Mac).

@computerpunc
Copy link
Contributor

computerpunc commented Aug 10, 2019

In addition,
I tried to push changes of #772 on a Mac that can run all tests without a problem.
Nevertheless, when trying to push via github client, I get the same error.

python-error

@dnalborczyk
Copy link
Collaborator Author

ah, ok. thanks for checking!

just noticed:

spawn python2 ENOENT

I guess it can't find python2 -> python2 --version probably fails.

@dnalborczyk
Copy link
Collaborator Author

@computerpunc should be fixed now

@computerpunc
Copy link
Contributor

computerpunc commented Aug 11, 2019

@dnalborczyk

I still cannot push any update because the push is rejected (from a Mac that runs the tests without any error). Github Desktop says that both python2 and python3 tests fail.
Where exactly do tests run when I try to push updates of a PR? Is it locally or remotely?

@dnalborczyk
Copy link
Collaborator Author

I still cannot push any update because the push is rejected (from a Mac that runs the tests without any error). Github Desktop says that both python2 and python3 tests fail.

it's just a git hook, you can run:

git push --no-verify

to get around it. not always recommended, but in this case a shortcut until we figure out what's wrong.
not sure how to do --no-verify with github desktop, and also not sure why github desktop behaves different in that case.

Where exactly do tests run when I try to push updates of a PR? Is it locally or remotely?

local only

@dnalborczyk
Copy link
Collaborator Author

dnalborczyk commented Aug 11, 2019

@computerpunc I can also add an ENV flag and/or npm script to skip 'flaky' tests - if that helps?

@computerpunc
Copy link
Contributor

I managed to push. I didn't need to use --no-verify because from the CLI the tests don't fail only with Github Desktop.

@remarcable
Copy link

Hello guys, it seems like the Readme has an incorrect npm install:
Instead of

npm i serverless@next --save-dev

it should probably be

npm i serverless-offline@next --save-dev

@dnalborczyk
Copy link
Collaborator Author

Hello guys, it seems like the Readme has an incorrect npm install:
Instead of

npm i serverless@next --save-dev

it should probably be

npm i serverless-offline@next --save-dev

thank you @lightningboss ! Fixed.

@computerpunc
Copy link
Contributor

@dnalborczyk

I see that for testing #781, you added another folder (python-big-json) in ./__tests__/integration.

That's going to be very messy if for each serverless.yml there will be a new folder.

serverless has the ability to consume configuration files with different names and locations.

In #732 and #778 (/src/__tests__/manual/websocket), I use the command sls offline --config ./config/serverless.$1.yml in which $1 can be main, authorizer or RouteSelection to run tests from .yml stored in the same folder.

I hope this helps.

@dnalborczyk
Copy link
Collaborator Author

@computerpunc

That's going to be very messy if for each serverless.yml there will be a new folder.

yeah, you are right. that was just a quick drop-in without thinking about the folder structure. that needs a proper rethinking and reorg for sure.

serverless has the ability to consume configuration files with different names and locations.

In #732 and #778 (/src/tests/manual/websocket), I use the command sls offline --config ./config/serverless.$1.yml in which $1 can be main, authorizer or RouteSelection to run tests from .yml stored in the same folder.

I hope this helps.

yeah, thanks for the tip!

I tried to use serverless.yml sub configuration files but couldn't get it to work at the time: https://github.com/dherault/serverless-offline/blob/master/__tests__/integration/handler/serverless.yml#L15

I might have to go back and have another look at it.

@verekia
Copy link

verekia commented Aug 18, 2019

EDIT: Created an issue instead of posting the whole thing here: #787 Sorry about that!

@dnalborczyk
Copy link
Collaborator Author

EDIT: Created an issue instead of posting the whole thing here: #787 Sorry about that!

@verekia no worries, you did the right thing. a new issue is better for tracking!

@computerpunc
Copy link
Contributor

@dnalborczyk

While working on #788, I found that Python 3 tests fail on AWS.
Run command (after sls deploy from folder ./__tests__/integration/python3):

npm --endpoint=https://xxxxxxxx.execute-api.us-east-1.amazonaws.com/dev run test __tests__/integration/python3/python3.test.js

Output:

> serverless-offline@6.0.0-alpha.15 test .../serverless-offline
> jest --verbose --silent --runInBand "__tests__/integration/python3/python3.test.js"

 FAIL  __tests__/integration/python3/python3.test.js
  Python 3 tests
    ✕ should work with python 3 (549ms)

  ● Python 3 tests › should work with python 3

    expect(received).toEqual(expected) // deep equality

    - Expected
    + Received

      Object {
    -   "message": "Hello Python 3!",
    +   "message": "Forbidden",
      }

      56 |       const response = await fetch(url)
      57 |       const json = await response.json()
    > 58 |       expect(json).toEqual(expected)
         |                    ^
      59 |     })
      60 |   })
      61 | })

      at Object.toEqual (__tests__/integration/python3/python3.test.js:58:20)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        1.31s, estimated 3s

@dnalborczyk
Copy link
Collaborator Author

dnalborczyk commented Aug 21, 2019

While working on #788, I found that Python 3 tests fail on AWS.

thanks for the info @computerpunc ! gonna check it out!

@dnalborczyk
Copy link
Collaborator Author

dnalborczyk commented Aug 22, 2019

@computerpunc It seems the npm endpoint stuff wasn't defined in that file. I fixed it. a57cc6d

@jonathonadams
Copy link
Contributor

jonathonadams commented Mar 17, 2020

Hi @dherault,

I can see alpha.68 has just been released, thank you.

Apologies if i'm wrong, but I think it was accidentally released to public?

i.e npm install serverless-offline installs 6.0.0-alpha.68
& npm install serverless-offline@next installs 6.0.0-alpha.67.

serverless-offline

@jormaechea
Copy link
Contributor

image

Indeed, alpha 68 has not been tagged as @next.. 😱

@dherault
Copy link
Owner

Nice catch, it does not matter since v6.0.0 will be released today!
Thanks everybody for such great feedbacks!

@dherault
Copy link
Owner

v6.0.0 🎉

@dherault dherault unpinned this issue Mar 19, 2020
@computerpunc
Copy link
Contributor

computerpunc commented Mar 22, 2020

I think I have found an issue with the introduction of the stage parameter into the URL.
Within API Gateway, an API with a URL such as https://**********.execute-api.eu-west-2.amazonaws.com/stage/foo/bar will pass the event.path property through to the Lambda as /foo/bar, stripping out the stage parameter.
The plugin currently passes through the full path to the Lambda handler e.g. /stage/foo/bar. This causes issues for Lambdas using Express, which handles the routing internally.
For now, I have worked around this by stripping out the stage (local in my specific use case) from the event before passing it on to Express:

event.path = event.path.replace(/^\/local/, "");

@dnalborczyk
After playing around a bit with alpha, I think that the default stage of serverless-offline should be a different one than the default stage of serverless. @jamesmbourne selection of local is good choice instead of dev.

@computerpunc that might be a good idea. let me think through it.

@dnalborczyk I don't know if you are still thinking about it but I did some digging and I don't thing it will be possible to implement because there's no way for serverless-offline to tell serverless that the default stage is local. The most I could get is to let serverrless inject by default the stage local to serverless-offline but event in this case, serverless treat the default stage as dev and not local.

My solution is always to run serverless-offline with the command sls offline -s local or even better have an alias for it.

@nathanchapman
Copy link

Can we please get a migration plan for the --stage variable? Any particular reason this was removed?

@designfrontier
Copy link
Contributor

@lee-reinhardt just started #937 to bring in the httpApi configs. Feel free to fill in any specific use cases over there and I'll see what I can do. Would love to get it in sooner rather then later so we can start using it for local dev and deploying out to httpApi

dustinsgoodman added a commit to dustinsgoodman/serverless-offline that referenced this issue Jun 12, 2020
Removing comments regarding the process.env.IS_OFFLINE variable as it has been removed by the plugin in the v6 alpha changes. See dherault#768 (comment) for more details.
dustinsgoodman added a commit to dustinsgoodman/serverless-offline that referenced this issue Jun 12, 2020
Removing comments regarding the process.env.IS_OFFLINE variable as it has been removed by the plugin in the v6 alpha changes. See dherault#768 (comment) for more details.
@mnapoli
Copy link

mnapoli commented Sep 23, 2020

Hi, I've been trying out the Docker feature but it seems like changes made to my code are not taken into account by the running sls offline command.

I have to stop sls offline and restart it, which is not really usable when developing (because we change the code a lot).

Is that the expected behavior or am I missing something? Thanks!

@Jordan-Eckowitz
Copy link

Hi, I've been trying out the Docker feature but it seems like changes made to my code are not taken into account by the running sls offline command.

I have to stop sls offline and restart it, which is not really usable when developing (because we change the code a lot).

Is that the expected behavior or am I missing something? Thanks!

For my team's setup we use nodemon to watch for file changes and restart sls offline. It works with Docker.

To make things easy you can add a script to package.json,
e.g. (in this example it watches for changes to js,py,json & yml files)

"api": "nodemon --ext js,py,json,yml --exec ./node_modules/.bin/serverless offline"

then you can simply call npm run api to start your API.

@mnapoli
Copy link

mnapoli commented Sep 23, 2020

@Jordan-Eckowitz thanks, so I should take it that it's the normal behavior?

Restarting serverless offline as well as the docker container on every code change is not really practical, I had a look through the code and it seems like the code is mounted (so should refresh), but maybe it's mounted from a copy of the codebase. I'm not sure if it would make sense to mount the actual code base (thus enabling immediate refresh).

@Jordan-Eckowitz
Copy link

@Jordan-Eckowitz thanks, so I should take it that it's the normal behavior?

Restarting serverless offline as well as the docker container on every code change is not really practical, I had a look through the code and it seems like the code is mounted (so should refresh), but maybe it's mounted from a copy of the codebase. I'm not sure if it would make sense to mount the actual code base (thus enabling immediate refresh).

@mnapoli you should be starting your docker container using docker-compose with volume mapping - that way it can dynamically pull those code changes into your container without restarting it

@mnapoli
Copy link

mnapoli commented Sep 24, 2020

@Jordan-Eckowitz wait are we talking about the same thing? Is this possible?

I'm talking about running serverless-offline with the new Docker feature to support custom runtimes. I have no control over the way docker is run by serverless offline.

@Jordan-Eckowitz
Copy link

@Jordan-Eckowitz wait are we talking about the same thing? Is this possible?

I'm talking about running serverless-offline with the new Docker feature to support custom runtimes. I have no control over the way docker is run by serverless offline.

@mnapoli apologies, I thought you were asking for general guidance on using Docker with serverless-offline - tbh I didn't know there was a new feature for it. I've setup the my API as described, with my own Docker setup, and it works seamlessly.

@Sunnova-ChristopherHuffman

@Jordan-Eckowitz wait are we talking about the same thing? Is this possible?
I'm talking about running serverless-offline with the new Docker feature to support custom runtimes. I have no control over the way docker is run by serverless offline.

@mnapoli apologies, I thought you were asking for general guidance on using Docker with serverless-offline - tbh I didn't know there was a new feature for it. I've setup the my API as described, with my own Docker setup, and it works seamlessly.

Would you be able to share any code repo that demostrate this type of setup? Im new to using docker and constantly starting/stopping containers when I make a simple code change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests