An extremely simple demo project showing an APIGateway -> Lambda -> SNS -> Lambda -> RDS pipeline, largely following this tutorial on AWS.
Microservice frameworks often use brokers to achieve async communication between microservices. On AWS, we can use SNS as our broker. In this demo, a client can POST a message to our API. A lambda function (snsProducer) then publishes a message on a topic (keyed by an ARN) on SNS. Another lambda (snsConsumer) is subscribed to the topic, and pulls the message off to be saved into a relational database.
API Gateway
+
| VPC
client | +-------------------------------------------------------+
+----------------+ | | +--------------+ +--------------+ +---------------+ |
| | | | |snsProducer | |snsConsumer | |RDS | |
| | | | | | | | | | |
| +---------------> | | | | | |
| | | | | + | | ^ +------------> | |
| <---------------+ | | | | | | | |
| | | | | | | | | <-----------+ | |
| | | | | | | | | | | | |
| | | | | | | | | | | | |
+----------------+ | | +--------------+ +--------------+ +---------------+ |
| +-------------------------------------------------------+
| +--------------------------------+
+ |SNS | | |
| v + |
| |
| |
+--------------------------------+
Stuff you want to customize from my setup is consolidated into awsConstants.js
.
Stuff we'll need to setup on AWS:
- User with the right permissions
- RDS (see also
aws.sh
for a sample rds initialization via cli) - VPC must have DNS hostname turned on, for EC2 (if you're using default VPC you should be good)
- EC2 for tunneling through to our RDS
- migration on RDS, via
setup.sql
- SNS
- Lambda & RDS should be in the same security group
- Lambda execution role
Stuff we'll need to setup locally:
- awscli
- aws credentials via
aws --configure
using credentials from user setup - node-lambda
npm install
insnsConsumer
andsnsProducer
directories- RDS credentials (environment vars) and SNS topic ARN (
awsConstants.js
)
To hit the endpoint, you can use ./snsProducer/bin/test.sh
from the project root.