Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Websocket #711

Merged
merged 87 commits into from Jul 4, 2019
Merged
Show file tree
Hide file tree
Changes from 84 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
c4811ca
demo websoket version
computerpunc May 8, 2019
e32b9fa
Added support for queryStringParameters
computerpunc May 14, 2019
1ea68e0
Bug fixes
computerpunc May 21, 2019
53e9eef
Run WebSocket endpoint on a different port (http port+1)
computerpunc May 23, 2019
602ac7b
GWAPI REST API suppoprt
computerpunc May 26, 2019
c4c6ea6
Changed the way to get function to post-to-connection
computerpunc May 27, 2019
10c377d
Default timeout is 1000ms and can set timeout value via --timeout=
computerpunc May 27, 2019
bca81cd
Minor Improved logs
computerpunc May 27, 2019
da8c749
Update README with new ways of sending messages to clients
computerpunc May 27, 2019
fe4a51d
Merge from upstream/master
computerpunc May 28, 2019
f527670
Leftovers from merge
computerpunc May 28, 2019
4ec82ad
Fix lint errors
computerpunc May 28, 2019
4e61aa5
Fix static AWS = { ... }
computerpunc May 28, 2019
289e8f5
Fix 2 lint errors
computerpunc May 28, 2019
ca34ccb
Removed require('aws-sdk/clients/apigatewaymanagementapi');
computerpunc May 29, 2019
2f3dfbd
Removed the need for require('serverless-offline').AWS
computerpunc May 30, 2019
f9600f8
Last commit leftovers
computerpunc May 30, 2019
f3cf815
Added context and event when calling the handler.
computerpunc Jun 1, 2019
4521b88
Added context and event for connect and disconnect
computerpunc Jun 3, 2019
96dbd16
Merge from master: hapi@18 & hapi-plugin-websocket@2
computerpunc Jun 4, 2019
1e559b7
hapi@18 and hapi-plugin-websocket@2 support
computerpunc Jun 4, 2019
1ea590e
Merge branch 'master' of https://github.com/dherault/serverless-offli…
computerpunc Jun 4, 2019
6638b73
Merge from master and fixes
computerpunc Jun 4, 2019
b51ddb5
added support for callback() in handler
computerpunc Jun 5, 2019
381355b
Moved websocket CreateXXX to websocketHelpers.js
computerpunc Jun 6, 2019
d45ea7a
Restructure manual_test_websocket to include more projects
computerpunc Jun 6, 2019
66ffe6a
Added serverless.yml with warning
computerpunc Jun 6, 2019
02ce0e7
Support for websocketsApiRouteSelectionExpression
computerpunc Jun 11, 2019
3759b06
Fix lint errors
computerpunc Jun 11, 2019
72a1fc4
Merge branch 'master' of https://github.com/dherault/serverless-offli…
computerpunc Jun 11, 2019
687f901
Merge leftovers
computerpunc Jun 11, 2019
b9b0eef
Update README
computerpunc Jun 11, 2019
b6c1095
Update README about WebSocket
computerpunc Jun 11, 2019
e855a4a
Export on exports
dnalborczyk Jun 12, 2019
53da9c1
Extract multi-value-headers function
dnalborczyk Jun 12, 2019
d802dcf
Generate collision resistent ids with cuid (project-wide)
dnalborczyk Jun 12, 2019
aa4f977
Add formatToClfTime function
dnalborczyk Jun 13, 2019
2df8ef8
Order props
dnalborczyk Jun 14, 2019
fd0637d
Remove unused options
dnalborczyk Jun 14, 2019
8a699ab
Add websocket port option
dnalborczyk Jun 14, 2019
7c34178
Fix spelling
dnalborczyk Jun 14, 2019
193e282
Separate ApiGateway and ApiGatewayWebSockets from Plugin
dnalborczyk Jun 15, 2019
536d1b3
Refactor events setup to offline class
dnalborczyk Jun 16, 2019
c51ea7e
Property order nits
dnalborczyk Jun 16, 2019
7b9c942
Spelling
dnalborczyk Jun 17, 2019
5aa2365
Require on top of file
dnalborczyk Jun 17, 2019
25832c8
Use Lambda context for websocket
dnalborczyk Jun 17, 2019
1386954
Add warning for experimental WebSocket support
dnalborczyk Jun 17, 2019
35a3616
Linting
dnalborczyk Jun 17, 2019
ca16506
Remove unused this.clients
dnalborczyk Jun 17, 2019
d631458
Rename variable
dnalborczyk Jun 17, 2019
0e3b4d8
Add websocket to package.json keywords
dnalborczyk Jun 18, 2019
741287b
Update deps
dnalborczyk Jun 18, 2019
d0fe602
Change minimum serverless peer dep to v1.39.0 for websocket support
dnalborczyk Jun 18, 2019
a9f4a09
Merge branch 'master' into websocket-fixes
dnalborczyk Jun 18, 2019
e54cdc3
Lint and rename folders
dherault Jun 22, 2019
fdbd07d
Merge branch 'master' into websocket-fixes
dherault Jun 22, 2019
c38a4bc
Display correct protocol on listen
dherault Jun 22, 2019
9247070
Add replay last request feature
dherault Jun 22, 2019
719a14e
Fix http/https --> HTTP
dherault Jun 22, 2019
209a95e
Use option to enable websocket feature
dherault Jun 22, 2019
6959108
Edit documentation
dherault Jun 22, 2019
e73983c
Fix travis build lint error
dherault Jun 22, 2019
d6eae11
Fix a test that didn't run correctly on AWS
computerpunc Jun 25, 2019
5e19fa6
Fixed test failing when running offline
computerpunc Jun 25, 2019
0f74255
Move to nodejs10.x runtime
computerpunc Jun 26, 2019
9ade958
Merge pull request #720 from computerpunc/websocket-fixes
dherault Jun 28, 2019
c5caceb
Merge branch 'master' into websocket-fixes
dherault Jun 30, 2019
7f56a18
Remove websocket feature toggling
dherault Jun 30, 2019
a77723e
remove --useWebsocket options
dherault Jun 30, 2019
9d84707
lint
dherault Jun 30, 2019
f489781
Update docs
dherault Jun 30, 2019
48c27d8
Merge branch 'master' into websocket-fixes
dherault Jul 1, 2019
1ba79f2
Merge branch 'master' into websocket-fixes
dherault Jul 2, 2019
27639ae
feat: add @connections DELETE route to close websocket
frsechet Jul 2, 2019
d5e14aa
fix: add missing printBlankLine method
frsechet Jul 2, 2019
e70db4b
chore: add tests for connection close
frsechet Jul 2, 2019
28266f2
Merge pull request #725 from frsechet/websocket-fixes
dherault Jul 2, 2019
b83c123
chore: if an error occurs during the $connect action, close the conne…
frsechet Jul 2, 2019
27e1e05
merge master
dnalborczyk Jul 2, 2019
b9b1d72
Merge pull request #726 from frsechet/fix/aws-difference
dherault Jul 2, 2019
c3fcc4c
Rename getUniqueId to createUniqueId
dnalborczyk Jul 3, 2019
b02e874
Merge master
dnalborczyk Jul 3, 2019
4f5ce66
Merge branch 'websocket-fixes' of https://github.com/dherault/serverl…
dnalborczyk Jul 3, 2019
c0bea4c
Remove apiGatewayUrl from websocket event
dnalborczyk Jul 3, 2019
f901507
fix: don't add body and content-type headers to sigv4 for delete call…
frsechet Jul 3, 2019
09d299a
Merge pull request #729 from frsechet/fix/tests-sigv4-delete-connection
dnalborczyk Jul 3, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 1 addition & 4 deletions .eslintignore
@@ -1,4 +1 @@
manual_test_nodejs
manual_test_python
manual_test_ruby
manual_test_websocket
**/node_modules
5 changes: 5 additions & 0 deletions .eslintrc.js
Expand Up @@ -8,6 +8,7 @@ const rules = {
'key-spacing': 'off',
'no-restricted-syntax': 'off',
'prefer-destructuring': 'off',
'one-var-declaration-per-line': ['error', 'always'],
semi: ['error', 'always'],
strict: 'off',
};
Expand All @@ -21,4 +22,8 @@ if (env.TRAVIS && platform === 'win32') {
module.exports = {
extends: 'dherault',
rules,
env: {
node: true,
mocha: true,
},
};
7 changes: 4 additions & 3 deletions .gitignore
Expand Up @@ -104,6 +104,7 @@ fabric.properties
# auto-generated tag files
tags
=======

.idea/
manual_test/.serverless

.idea/
.serverless
.dynamodb
30 changes: 30 additions & 0 deletions README.md
Expand Up @@ -28,6 +28,7 @@ This plugin is updated by its users, I just do maintenance and ensure that PRs a
* [Custom headers](#custom-headers)
* [Environment variables](#environment-variables)
* [AWS API Gateway features](#aws-api-gateway-features)
* [WebSocket](#websocket)
* [Usage with Webpack](#usage-with-webpack)
* [Velocity nuances](#velocity-nuances)
* [Debug process](#debug-process)
Expand Down Expand Up @@ -75,6 +76,7 @@ All CLI options are optional:
--location -l The root location of the handlers' files. Defaults to the current directory
--host -o Host name to listen on. Default: localhost
--port -P Port to listen on. Default: 3000
--websocketPort WebSocket port to listen on. Default: 3001
--stage -s The stage used to populate your templates. Default: the first stage found in your project.
--region -r The region used to populate your templates. Default: the first region for the first stage found.
--noTimeout -t Disables the timeout feature.
Expand Down Expand Up @@ -359,6 +361,34 @@ resources:

To disable the model validation you can use `--disableModelValidation`.

## WebSocket

:warning: *This is an experimental functionality. Please report any bugs or missing features. PRs are welcome.*

Usage in order to send messages back to clients:

`POST http://localhost:{websocketPort}/@connections/{connectionId}`

Or,

```js
const apiGatewayManagementApi = new AWS.ApiGatewayManagementApi({
apiVersion: '2018-11-29',
endpoint: event.apiGatewayUrl || `${event.requestContext.domainName}/${event.requestContext.stage}`,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@computerpunc are you sur this line is correct? Wouldn't the endpoint be localhost:3001?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, unless you have another idea how to inject address (it can be localhost:3001, localhost:3002, etc.) to the function.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what do we input in the docs:
endpoint: process.env.IS_OFFLINE ? 'http://localhost:3001' : undefined
This works well for lambda.invoke.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dherault, @dnalborczyk, @frsechet and others:

Let's start from the beginning:
If AWS would have injected by default event.requestContext.serverPort, we wouldn't have this discussion because all information would be at hand.
I could have gone with injecting event.requestContext.serverPort but I thought that it was too much to handle all the cases (specific port or default, with http or not) so I went with apiGatewayUrl. Any other suitable name will be fine of course.
The drawback, with process.env.IS_OFFLINE ? 'http://localhost:3001' : is that the port is hardcoded.
What happens if you want to run 2 local instances of the same app?

So we have the following options:

  1. Hardcoded - http://localhost:3001.
  2. Inject port in event- event.requestContext.serverPort.
  3. Inject port in environment - process.env.SERVER_PORT.
  4. Use current implementation - event.apiGatewayUrl.
  5. Use current implementation with renaming apiGatewayUrl to something else.

What do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on the end of the day it's really up to the user what to use. I personally did use #3 (env, but for the entire endpoint url), but I might switch to this:

serverless in the plugin implementation used: https://github.com/serverless/serverless-websockets-plugin/blob/master/example/src/websocket-client.js#L36 as well as https://www.freecodecamp.org/news/real-time-applications-using-websockets-with-aws-api-gateway-and-lambda-a5bb493e9452/

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@computerpunc I couldn't find any information on apiGatewayUrl. is that something you added to the event? if so, we might want to remove it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dnalborczyk
https://${config.requestContext.domainName}/${config.requestContext.stage}
is not going to work when running offline. This is what the whole discussion is about. It will not work because the port is missing. So the question is how to get the port. Above, I wrote 5 options. Option (1) is my least favorite. Any other is fine. Whatever you find easier to use and more esthetic. No problem to go with hardcoded now (that is remove apiGarewayUrl) and add whatever needed support later.

Since the original PR, apiGarewayUrl is injected into the event object (AWS doesn’t do so).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, thanks. then it should be removed, although helpful, it would be quite confusing to add additional properties on the event, other than the aws provided ones.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as the docs are concerned, just go with a simple approach, hardcoded. whatever users are deciding to use is their individual choice. (env, hardcoded, etc.)

});

apiGatewayManagementApi.postToConnection({
ConnectionId: ...,
Data: ...,
});
```
dherault marked this conversation as resolved.
Show resolved Hide resolved

Where the `event` is received in the lambda handler function.
dherault marked this conversation as resolved.
Show resolved Hide resolved

There's support for [websocketsApiRouteSelectionExpression](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api-selection-expressions.html) in it's basic form: `$request.body.x.y.z`, where the default value is `$request.body.action`.

Authorizers and wss:// are currectly not supported in this feature.

## Usage with Webpack

Use [serverless-webpack](https://github.com/serverless-heaven/serverless-webpack) to compile and bundle your ES-next code
Expand Down
7 changes: 3 additions & 4 deletions manual_test_nodejs/handler.js
@@ -1,4 +1,3 @@
'use strict';

module.exports.hello = (event, context, callback) => {
const response = {
Expand Down Expand Up @@ -28,7 +27,7 @@ module.exports.rejectedPromise = (event, context, callback) => {
callback(null, response);
};

module.exports.authFunction = (event, context, callback) => {
module.exports.authFunction = (event, context) => {
context.succeed({
principalId: 'xxxxxxx', // the principal user identification associated with the token send by the client
policyDocument: {
Expand Down Expand Up @@ -98,6 +97,6 @@ module.exports.pathParams = (event, context, cb) => {
cb(null, response);
};

module.exports.failure = (event, context, cb) => {
throw new Error('Unexpected error!')
module.exports.failure = () => {
throw new Error('Unexpected error!');
};
1 change: 0 additions & 1 deletion manual_test_nodejs/subprocess.js
@@ -1,4 +1,3 @@
'use strict';

const { exec } = require('child_process');

Expand Down
1 change: 1 addition & 0 deletions manual_test_websocket/.gitignore
@@ -0,0 +1 @@
/**/serverless.yml
25 changes: 11 additions & 14 deletions manual_test_websocket/README.md
Expand Up @@ -8,6 +8,7 @@ Set AWS credentials, e.g.: `export AWS_PROFILE=...`

To start AWS DynamoDB locally (can run only after first deploying locally): `sls dynamodb install` `sls dynamodb start`

### In either main or RouteSelection folder the following:

## Deploying locally

Expand All @@ -26,28 +27,24 @@ To start AWS DynamoDB locally (can run only after first deploying locally): `sls

## Testing on AWS

`npm --endpoint={WebSocket endpoint URL on AWS} run test`
`npm --endpoint={WebSocket endpoint URL on AWS} --timeout={timeout in ms} run test`


## Usage Assumption - In order to send messages back to clients
`const newAWSApiGatewayManagementApi=(event, context)=>{`
## Usage in order to send messages back to clients

`POST http://localhost:3001/@connections/{connectionId}`

` const endpoint=event.requestContext.domainName+'/'+event.requestContext.stage;`
Or,

` const apiVersion='2018-11-29';`
`let endpoint=event.apiGatewayUrl;`

` let API=context.API;`
`if (!endpoint) endpoint = event.requestContext.domainName+'/'+event.requestContext.stage;`

` if (!process.env.IS_OFFLINE) {`
`const apiVersion='2018-11-29';`

` API = require('aws-sdk');`
`const apiGM=new API.ApiGatewayManagementApi({ apiVersion, endpoint });`

` require('aws-sdk/clients/apigatewaymanagementapi');`
`apiGM.postToConnection({ConnectionId, Data});`

` }`

` return new API.ApiGatewayManagementApi({ apiVersion, endpoint });`

`};`


31 changes: 31 additions & 0 deletions manual_test_websocket/RouteSelection/handler.js
@@ -0,0 +1,31 @@
const AWS = require('aws-sdk');

const successfullResponse = {
statusCode: 200,
body: 'Request is OK.',
};

module.exports.echo = async (event, context) => {
const action = JSON.parse(event.body);

await sendToClient(action.message, event.requestContext.connectionId, newAWSApiGatewayManagementApi(event, context));

return successfullResponse;
};

const newAWSApiGatewayManagementApi = event => {
let endpoint = event.apiGatewayUrl;

if (!endpoint) endpoint = `${event.requestContext.domainName}/${event.requestContext.stage}`;
const apiVersion = '2018-11-29';

return new AWS.ApiGatewayManagementApi({ apiVersion, endpoint });
};

const sendToClient = (data, connectionId, apigwManagementApi) => {
// console.log(`sendToClient:${connectionId}`);
let sendee = data;
if (typeof data === 'object') sendee = JSON.stringify(data);

return apigwManagementApi.postToConnection({ ConnectionId: connectionId, Data: sendee }).promise();
};