How to use queues in your serverless app
In this example we will look at how to use SQS to create a queue in our serverless app using SST. We’ll be creating a simple queue system.
Requirements
- Node.js 16 or later
- We’ll be using TypeScript
- An AWS account with the AWS CLI configured locally
Create an SST app
Let’s start by creating an SST app.
$ npx create-sst@latest --template=base/example queue
$ cd queue
$ npm install
By default, our app will be deployed to the us-east-1
AWS region. This can be changed in the sst.config.ts
in your project root.
import { SSTConfig } from "sst";
export default {
config(_input) {
return {
name: "queue",
region: "us-east-1",
};
},
} satisfies SSTConfig;
Project layout
An SST app is made up of two parts.
-
stacks/
— App InfrastructureThe code that describes the infrastructure of your serverless app is placed in the
stacks/
directory of your project. SST uses AWS CDK, to create the infrastructure. -
packages/functions/
— App CodeThe code that’s run when your API is invoked is placed in the
packages/functions/
directory of your project.
Adding SQS Queue
Amazon SQS is a reliable and high-throughput message queuing service. You are charged based on the number of API requests made to SQS. And you won’t get charged if you are not using it.
Replace the stacks/ExampleStack.ts
with the following.
import { StackContext, Queue, Api } from "sst/constructs";
export function ExampleStack({ stack }: StackContext) {
// Create Queue
const queue = new Queue(stack, "Queue", {
consumer: "packages/functions/src/consumer.main",
});
}
This creates an SQS queue using Queue
. And it has a consumer that polls for messages from the queue. The consumer function will run when it has polled 1 or more messages.
Setting up the API
Now let’s add the API.
Add this below the Queue
definition in stacks/ExampleStack.ts
.
// Create the HTTP API
const api = new Api(stack, "Api", {
defaults: {
function: {
// Bind the table name to our API
bind: [queue],
},
},
routes: {
"POST /": "packages/functions/src/lambda.main",
},
});
// Show the API endpoint in the output
stack.addOutputs({
ApiEndpoint: api.url,
});
Our API simply has one endpoint (the root). When we make a POST
request to this endpoint the Lambda function called main
in packages/functions/src/lambda.ts
will get invoked.
We’ll also bind our queue to our API.
Adding function code
We will create two functions, one for handling the API request, and one for the consumer.
Replace the packages/functions/src/lambda.ts
with the following.
export async function main() {
console.log("Message queued!");
return {
statusCode: 200,
body: JSON.stringify({ status: "successful" }),
};
}
Add a packages/functions/src/consumer.ts
.
import { SQSEvent } from "aws-lambda";
export async function main(event: SQSEvent) {
const records: any[] = event.Records;
console.log(`Message processed: "${records[0].body}"`);
return {};
}
Now let’s test our new API.
Starting your dev environment
SST features a Live Lambda Development environment that allows you to work on your serverless apps live.
$ npm run dev
The first time you run this command it’ll take a couple of minutes to deploy your app and a debug stack to power the Live Lambda Development environment.
Deployed:
ExampleStack
ApiEndpoint: https://3vi820odbc.execute-api.us-east-1.amazonaws.com
The ApiEndpoint
is the API we just created.
Let’s test our endpoint. Run the following in a new terminal.
$ curl -X POST https://3vi820odbc.execute-api.us-east-1.amazonaws.com
This makes a POST request to our API. You should see Message queued!
in the sst dev
terminal.
Sending message to our queue
Now let’s send a message to our queue.
Replace the packages/functions/src/lambda.ts
with the following.
import AWS from "aws-sdk";
import { Queue } from "sst/node/queue";
const sqs = new AWS.SQS();
export async function main() {
// Send a message to queue
await sqs
.sendMessage({
// Get the queue url from the environment variable
QueueUrl: Queue.Queue.queueUrl,
MessageBody: JSON.stringify({ ordered: true }),
})
.promise();
console.log("Message queued!");
return {
statusCode: 200,
body: JSON.stringify({ status: "successful" }),
};
}
Here we are getting the queue url from the environment variable, and then sending a message to it.
Let’s install the aws-sdk
package in the packages/functions/
folder.
$ npm install aws-sdk
Now if you hit our API again.
$ curl -X POST https://3vi820odbc.execute-api.us-east-1.amazonaws.com
You should see Message processed: "{"ordered":true}"
printed out in the sst dev
terminal.
Deploying to prod
To wrap things up we’ll deploy our app to prod.
$ npx sst deploy --stage prod
This allows us to separate our environments, so when we are working in dev
, it doesn’t break the API for our users.
Cleaning up
Finally, you can remove the resources created in this example using the following commands.
$ npx sst remove
$ npx sst remove --stage prod
Conclusion
And that’s it! We’ve got a completely serverless queue system. Check out the repo below for the code we used in this example. And leave a comment if you have any questions!
Example repo for reference
github.com/sst/sst/tree/master/examples/queueFor help and discussion
Comments on this exampleMore Examples
APIs
-
REST API
Building a simple REST API.
-
WebSocket API
Building a simple WebSocket API.
-
Go REST API
Building a REST API with Golang.
-
Custom Domains
Using a custom domain in an API.
Web Apps
-
React.js
Full-stack React app with a serverless API.
-
Next.js
Full-stack Next.js app with DynamoDB.
-
Vue.js
Full-stack Vue.js app with a serverless API.
-
Svelte
Full-stack SvelteKit app with a serverless API.
-
Gatsby
Full-stack Gatsby app with a serverless API.
-
Angular
Full-stack Angular app with a serverless API.
Mobile Apps
GraphQL
Databases
-
DynamoDB
Using DynamoDB in a serverless API.
-
MongoDB Atlas
Using MongoDB Atlas in a serverless API.
-
PostgreSQL
Using PostgreSQL and Aurora in a serverless API.
-
CRUD DynamoDB
Building a CRUD API with DynamoDB.
-
PlanetScale
Using PlanetScale in a serverless API.
Authentication
Using SST Auth
-
Facebook Auth
Adding Facebook auth to a full-stack serverless app.
-
Google Auth
Adding Google auth to a full-stack serverless app.
Using Cognito Identity Pools
-
Cognito IAM
Authenticating with Cognito User Pool and Identity Pool.
-
Facebook Auth
Authenticating a serverless API with Facebook.
-
Twitter Auth
Authenticating a serverless API with Twitter.
-
Auth0 IAM
Authenticating a serverless API with Auth0.
Using Cognito User Pools
-
Cognito JWT
Adding JWT authentication with Cognito.
-
Auth0 JWT
Adding JWT authentication with Auth0.
-
Google Auth
Authenticating a full-stack serverless app with Google.
-
GitHub Auth
Authenticating a full-stack serverless app with GitHub.
-
Facebook Auth
Authenticating a full-stack serverless app with Facebook.
Async Tasks
-
Cron
A simple serverless Cron job.
-
Pub/Sub
A simple pub/sub system with SNS.
-
Resize Images
Automatically resize images uploaded to S3.
-
Kinesis data streams
A simple Kinesis Data Stream system.
-
EventBus
A simple EventBridge system with EventBus.
Editors
-
Debug With VS Code
Using VS Code to debug serverless apps.
-
Debug With WebStorm
Using WebStorm to debug serverless apps.
-
Debug With IntelliJ
Using IntelliJ IDEA to debug serverless apps.
Monitoring
Miscellaneous
-
Lambda Layers
Using the @sparticuz/chromium layer to take screenshots.
-
Middy Validator
Use Middy to validate API request and responses.