Skip to content

A set of functions to help sending bulk messages in sequence or parallel to AWS SQS.

License

Notifications You must be signed in to change notification settings

montumodi/sqs-bulk-loader

Repository files navigation

AWS SQS Bulk Loader

A set of functions to help sending bulk messages in sequence or parallel to AWS SQS.

Migration steps from Version 3 to Version 4

  • Add @aws-sdk/client-sqs as peer dependency instead of aws-sdk
  • Rest of the code should work as it is. The response structure has changed slightly. More information can be found here

Known Vulnerabilities Coverage Status Build Status

NPM

How to install

npm install sqs-bulk-loader

@aws-sdk/client-sqs is peer dependency for this package. Make sure it is installed.

To use aws-sdk version 2, please use version 3 of this package.

npm install sqs-bulk-loader@3

Running the tests

npm test

Getting started

Basic syntax is:

const {sendBatchedMessages, sendBatchedMessagesInParallel} = require("sqs-bulk-loader")();

const messages = [
  {
    "Id": "1",
    "MessageBody": '{"key1": "value1"}'
  },
  {
    "Id": "2",
    "MessageBody": '{"key2": "value2"}'
  }
];

// this needs to be in async function
const response = await sendBatchedMessages("someQueueUrl", messages);
console.log(response)

// OR you can use normal promise style as well.
sendBatchedMessages("someQueueUrl", messages)
  .then(response => console.log(response));

In case you need to inject the @aws-sdk/client-sqs with custom settings. It can be done like this

const {SQSClient} = require("@aws-sdk/client-sqs");
const sqsClient = new SQSClient({"region": "eu-west-2"});
const {sendBatchedMessages, sendBatchedMessagesInParallel} = require("sqs-bulk-loader")(sqsClient);

API

sendBatchedMessages

sendBatchedMessages(queueUrl, messages) ⇒ Promise

Function - Sends the messages passed in batch of 10 sequentially

Returns: Promise - - promise which resolves on success and rejects on error

Param Type Default Description
queueUrl String SQS queue url
messages Array Array of messages as per sendMessageBatch's params

More details - https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sqs/interfaces/sendmessagebatchcommandinput.html

sendBatchedMessagesInParallel

sendBatchedMessagesInParallel(queueUrl, messages) ⇒ Promise

Function - Sends the messages passed in batch of 10 parallely

Returns: Promise - - promise which resolves on success and rejects on error

Param Type Default Description
queueUrl String SQS queue url
messages Array Array of messages as per sendMessageBatch's params
[options] Object {"batchSize": 10} Optional object containing extra properties which will be passed to function. Currently it supports batchSize integer to control how many parallel requests to spawn. Default is 10

More details - https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sqs/interfaces/sendmessagebatchcommandinput.html