Support for HttpApi $default route, $default stage and 2.0 payloads

[apancutt]

Description

Fixes #1118 but also addresses other issues found with httpApi events:

• Default payload format is now 2.0 (NB: also implemented by Default httpApi payload to 2.0 #1126 which does not include required fixes to 2.0 event payloads)
• Support for $default route
• Support for $default stage (Serverless always deploys to $default stage which should never be prepended to URL)
• Support for CORS configuration which is provided in provider.httpApi (reverted in favor of Add support for HttpApi cors configuration #1153)
• Fix LambdaProxyIntegrationEvent.resource which should be the route key (for httpApi events only)
• Fix LambdaProxyIntegrationEventV2.routeKey which should be the route key
• Fix LambdaProxyIntegrationEventV2.rawPath which should be obtained from the requested URL (not the route)
• Optimize LambdaProxyIntegrationEventV2.rawQueryString and LambdaProxyIntegrationEventV2.queryStringParameters to use existing URL instance provided in #request.

Motivation and Context

It is currently not possible to define $default routes for httpApi events. The current workaround requires inconsistent implementation between Serverless Offline and AWS (using a {proxy+} hack locally).

The default event payload should reflect the default payload provisioned by Serverless/AWS.

The 2.0 payload format is currently wrong.

How Has This Been Tested?

Compared event payloads (1.0 and 2.0) received on AWS against those received via Serverless Offline for both $default route and named routes.

Tested locally - payloads 1.0 and 2.0 are both handled successfully for named routes and $default route (defined in serverless.yml as { httpApi: '*' }).

Updated related tests by removing stage prefix which is not used by Serverless (it unconditionally uses the $default stage).

[bryanvaz]

LOL, @apancutt, I was just about to patch the path params for httpApi events when I found your fix. The code looks like it should work, but I haven't had a chance to put it through all the paces.

Few quick questions:

• I assume it works for path params path: /path/with/param/{aParam}?
• I assume it works for path params path: /path/with/param/{proxy+}?
• Follows the priorities for all the other route possibilities listed in the table AWS docs https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-routes.html#http-api-develop-routes.evaluation?

Do you have time to add tests those three route scenarios? They are probably:

• path: {$default} in isolation
• path: {$default} in with another higher priority path
• path: /path/with/param/{aParam}
• path: /path/with/param/{proxy+}
  (They should be copies of each other with just the path changed)

@frozenbonito / @dherault would you put those path tests under integration or endToEnd?

I'm guessing integration because it a nuance just for httpApi events.

Cheers,
Bryan

[bryanvaz]

@apancutt
Hey Adam (and for @dherault when you release),

Just a BREAKING CHANGE note for your PR - specifically the change:

Support for $default stage (Serverless always deploys to $default stage which should never be prepended to URL)

This changes the previous behaviour of inserting the serverless stage into the url when using it in offline mode. This affects all httpApi events, regardless of whether or not a $default is defined.

This will break any integration tests that use sls offline and expects the current endpoint construction for httpApi events.

Essentially current httpApi offline endpoints that were (for dev stage):
*http://0.0.0.0:3000/dev/a/test
will now be:
*http://0.0.0.0:3000/a/test

I'm just starting to work through the PR in an active project, so I'll report any other findings as they come up.

Bryan

[jakemwood]

Hi there -- I started using this PR because we need v2 payloads in order to use serverless-offline. I just wanted to point out one notable difference between using this package and the real API Gateway. It seems the API Gateway always makes incoming HTTP header names lower-cased, but serverless-offline seems to pass them through as-is.

I don't know if that's an issue that needs to be addressed with this PR (is this a specific trait of the v2 gateway requests?), or if this is a larger issue with this package. I'd be happy to help wherever I can with this PR, though, we need it! 😄 Thanks!

[nponeccop]

It seems the API Gateway always makes incoming HTTP header names lower-cased,
but serverless-offline seems to pass them through as-is.

I've just reported this issue independently as #1207

[nponeccop]

Is there anything that prevent this from getting merged?

[bryanvaz]

@nponeccop there is nothing preventing this from from getting merged.
@frozenbonito, @medikoo, and @pgrzesik can we push this out ASAP?

Currently using this for production development and it's a bit awkward referencing a PR branch...

Cheers,
Bryan

[pgrzesik]

Hello @bryanvaz - I'm just getting up to speed with the project so I don't feel confident yet in fully assessing the change - you mentioned a breaking change in one of the previous comments - could you explain a bit more? Is that breaking change necessary?

[apancutt]

@pgrzesik the breaking change is that the stage name is never prepended to the URL. This is because Serverless will always deploy to the $default stage which is not prefixed (unlike REST APIs).

It's possible that many/most users will already be disabling the stage prefix using the noPrependStageInUrl option for consistency with the deployed service but, for those that haven't, this is a breaking change.

[apancutt]

In fact, all of the changes could be considered breaking given the right set of circumstances, except for the last one on the list above.

Possibly warrants a major version bump.

[bryanvaz]

@pgrzesik the breaking change is that the stage name is never prepended to the URL. This is because Serverless will always deploy to the $default stage which is not prefixed (unlike REST APIs).

It's possible that many/most users will already be disabling the stage prefix using the noPrependStageInUrl option for consistency with the deployed service but, for those that haven't, this is a breaking change.

Exactly, it's less a breaking change in the sense that serverless-offline was working correctly and this PR will break anything that uses httpApi routes. It is more of the situation where serverless-offline was incorrectly adding stage prefixes for $default routes and requiring users to find a workaround, and this PR eliminates the need for a workaround; however, as with any workaround patch, there is a chance that local apps with such a patch may break when the need for that patch is eliminated.

[pgrzesik]

Thanks @bryanvaz for detailed explanation 🙇 I believe we're safe to merge this PR and it will be a good idea to release it as a part of next major release to avoid potentially breaking local workflows, even though it's actually fixing the behavior of httpApi. 👍

[pgrzesik]

Hello @apancutt - it looks like unfortunatelly, there was a slight regression introduced with this PR which causes one CORS-related tests to fail, do you think you'd be able to take a look at it? Example: https://github.com/dherault/serverless-offline/runs/2391560450?check_suite_focus=true