SQS, EventBridge and LocalStack

Photo by Kaleidico on Unsplash

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple the components of a cloud application. However, during development and testing, it can be costly and sometimes inconvenient to interact with the actual AWS SQS service. This is where LocalStack comes to the rescue. LocalStack is a fantastic tool for simulating AWS services locally, enabling you to develop and test your applications without incurring AWS costs.

In this article, we’ll explore how to use LocalStack to create an SQS queue, send messages to it, and receive messages from it using Node.js. We’ll walk through the necessary setup, including creating an Event bus, SQS queue, setting up event rules, and creating producers and consumers for your local development environment. With LocalStack, you can simulate the entire process without relying on the actual AWS infrastructure.

Prerequisites:

Before we dive into the code examples, make sure you have the following prerequisites:

• Docker installed on your machine
• Node.js and npm (Node Package Manager) installed
• AWS CLI (for configuring LocalStack)

Setting Up LocalStack:

To get started, we’ll set up LocalStack to mimic AWS services locally. Create a docker-compose.yml file with the following content:

version: 3
services:
  localstack:
    image: localstack/localstack:0.12.9.1
    networks:
      - localstack
    ports:
      - "4577:4566"
    environment:
      - SERVICES=events,sqs
      - DATA_DIR=/tmp/localstack/data
      - DEBUG=true
      - DOCKER_HOST=unix:///var/run/docker.sock
      - AWS_ACCESS_KEY_ID=my_key
      - AWS_SECRET_ACCESS_KEY=my_secret
      - LAMBDA_EXECUTOR=docker
    volumes:
      - "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
      - /var/run/docker.sock:/var/run/docker.sock
networks:
  localstack:
    name: localstack

This Docker Compose file sets up a LocalStack instance with the necessary environment variables to mimic AWS services. Save it to a file, and then run docker-compose up to start LocalStack.

Creating an SQS Queue:

Next, let’s create an Eventbus and SQS queue using the AWS CLI with the LocalStack endpoint:

# Create event bus
aws events --endpoint-url http://localhost:4577 create-event-bus --name custom-event-bus --region us-east-1
# Create an SQS queue
aws sqs --endpoint-url http://localhost:4577 create-queue --queue-name service-a --region us-east-1
# List available queues
aws sqs --endpoint-url http://localhost:4577 list-queues

Now that we have our local SQS queue set up, let’s proceed to create an event rule and a target for it.

Creating an Event Rule and Target:

We need to create an event rule that specifies what kind of events will trigger actions in our SQS queue. In this example, we’re creating a rule that matches events with a “greeting” detail type:

# Create an event rule
aws events --endpoint-url http://localhost:4577 put-rule --name send-events-to-service-a --event-bus-name custom-event-bus --event-pattern '{"detail-type": ["greeting"]}'

Finally, we’ll create a target for this event rule, which will send matched events to our SQS queue:

# Create a target for the event rule
aws events --endpoint-url http://localhost:4577 put-targets --event-bus-name custom-event-bus --rule send-events-to-service-a --targets "Id"="service-a","Arn"="arn:aws:sqs:us-east-1:000000000000:service-a"

With these infrastructure pieces in place, you’re ready to start building the code for your producer and consumer.

Creating an Event Producer:

Now that we have set up our LocalStack environment and event rules, it’s time to create an event producer. In this section, we’ll explore how to use Node.js to send events to our custom event bus within LocalStack.

We’ll use the @aws-sdk/client-eventbridge package to interact with EventBridge programmatically. If you haven't already, you can install this package using npm:

npm install @aws-sdk/client-eventbridge

Here’s the Node.js code to send events to your LocalStack-powered EventBridge:

const {
  EventBridgeClient,
  PutEventsCommand,
} = require("@aws-sdk/client-eventbridge");

async function putEvent(
  source = "service-a",
  detailType = "greeting",
  resources = []
) {
  const client = new EventBridgeClient({
    region: "us-east-1",
    endpoint: "http://localhost:4577", // LocalStack endpoint
  });
  const response = await client.send(
    new PutEventsCommand({
      Entries: [
        {
          EventBusName: "custom-event-bus",
          Detail: JSON.stringify({ greeting: "Hello there." }),
          DetailType: detailType,
          Resources: resources,
          Source: source,
        },
      ],
    })
  );
  return response;
}

setInterval(() => {
  putEvent().then((response) => {
    console.log("Event sent", response);
  });
}, 5000);

Creating an Event Consumer:

The consumer code provided below will continuously poll the queue for messages and process them when they arrive.

Here’s the Node.js code for the event consumer:

// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");
var sqs = new AWS.SQS({ apiVersion: "2012-11-05" });
var queueURL = "http://localhost:4577/000000000000/service-a";
var params = {
  AttributeNames: ["SentTimestamp"],
  MaxNumberOfMessages: 1,
  MessageAttributeNames: ["All"],
  QueueUrl: queueURL,
  VisibilityTimeout: 20,
  WaitTimeSeconds: 0,
};
// Function to receive and process messages
function receiveMessage() {
  sqs.receiveMessage(params, function (err, data) {
    if (err) {
      console.log("Receive Error", err);
    } else if (data.Messages) {
      console.log("Message received", data.Messages[0].Body);
      // Delete the processed message from the queue
      var deleteParams = {
        QueueUrl: queueURL,
        ReceiptHandle: data.Messages[0].ReceiptHandle,
      };
      sqs.deleteMessage(deleteParams, function (err, data) {
        if (err) {
          console.log("Delete Error", err);
        } else {
          console.log("Message Deleted", data);
        }
      });
    }
  });
}

setInterval(() => {
  console.log("Receiving message...");
  receiveMessage();
}, 5000);

LocalStack simplifies the development process and reduces AWS costs, making it a valuable tool for developers and teams working with AWS services.

As you further explore and develop your event-driven applications, consider fine-tuning the code and adjusting the settings to meet your specific use cases. Whether you’re building microservices or working on larger projects, LocalStack can be a valuable addition to your development toolkit, making AWS service simulation an efficient and cost-effective process.

Happy coding!