# Amazon SQS

Amazon SQS (opens new window) is a web service offered by Amazon Web Services, that provides fully managed messages queues for microservices, distributed systems and serverless applications.

# API version

The Amazon SQS connector uses the Amazon SQS REST API (opens new window).

# How to connect to SQS on Workato

The SQS connector allows two types of authentication: IAM role and Access key. Workto recommends using IAM role authentication.

# Input fields

Field Description
Connection name Give this connection a unique name that identifies which SQS instance it is connected to.
Authorization type IAM role or Access key
Access key ID (Access key only) The ID of the user.
Secret access key (Access key only) The secret of the user.
IAM role ARN (IAM role only) The IAM role ARN. Find out how to retrieve this here.

Note: Workato will generate a unique external id (for example, workato-user-84762). This value is different for every Workato user and must be provided when creating an IAM role in SQS.
Region Provide the region for this S3 account.
Version The SQS version. Today only 2012-11-05 is available.
Service Your SQS queue service, defined in the queue ARN. If your queue ARN is arn:aws:sqs-fips:us-east-1:123456789:bennett_standard_q, use sqs-fips as the service.

# How to retrieve IAM role ARN

Follow these steps to retrieve the Role ARN required for the connection setup. Remember to use the Workato generated external ID found in the connection page.

1

Navigate to My Security Credentials.

My Security Credentials

2

Select Roles > Create role.

Create role

3

Select Another AWS account. Input Workato's Amazon SQS Account ID (353360065216).

Workato Amazon S3 Account ID

4

Select Require external ID and provide the Workato generated External ID.

Every Workato user will have a unique External id (for example, workato-user-84762). You can find this value in the IAM role ARN portion of the connection setup.

Require External ID

5

Select proper permissions for Workato to run automation in your Amazon SQS. Read below to see what permissions are required.

6

(optional) If you are using object taggings, select an appropriate tag for this IAM role.

Add tag

7

Give this IAM Role an appropriate name & description.<

Workato recommends that role name avoids using a non-guessable resource-id in the urn and does not include the external id.

Review role

8

The IAM Role is now created. Select the role.

Select IAM role

9

Copy the Role ARN. You will need to use this in the connection setup when creating an Amazon S3 connection in Workato.

Copy role ARN

# Permissions

You may use the AmazonSQSFullAccess policy for this IAM role. At a minimum the SQS connector requires these permissions:

  • sqs:ReceiveMessage
  • sqs:DeleteMessage
  • sqs:ListQueues
  • sqs:GetQueueAttributes
  • sqs:SendMessage

For establishing connection, we use "List Queues" to test connection, but if you have access only to a single queue or to a few specific queues, the connection will still work and you can proceed to perform operations on those queue by manually adding the queue name in the actions.

# Terminology

The terminology (opens new window) for basic Amazon SQS objects can be found below.

# Queues

There are two types of queues in SQS:

  • Standard Queues - The default queue type which support at-least-once message delivery. Standard queues provide best-effort ordering which ensures that messages are generally delivered in the same order as they're sent.
  • FIFO Queues - FIFO (First-In-First-Out) queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can't be tolerated.

Only standard queues are supported by the Workato SQS connector today.

# Messages

Amazon SQS lets you include structured metadata (such as timestamps, geospatial data, signatures, and identifiers) with messages using message attributes. Each message can have up to 10 attributes. Message attributes are optional and separate from the message body (however, they are sent alongside it). Workato supports both message bodies and message attributes.

When a consumer receives and processes a message from a queue, the message remains in the queue. Amazon SQS doesn't automatically delete the message. Because Amazon SQS is a distributed system, there's no guarantee that the consumer actually receives the message (for example, due to a connectivity issue, or due to an issue in the consumer application). To aid in this, the Workato SQS connector allows you to automaticaly delete messages when you retrieve them in triggers. Alternatively, you may also use the Delete messages action to delete messages after you have processed them in your recipe.

# Short vs Long polling

Due to SQS's distributed nature, a single request to SQS may not retrieve all messages in the queue. For example, a request to retrieve 10 messages may only retrieve 9 messages when there could be hundreds of messages in the queue. This is the result of SQS sampling a subset of servers holding messages to return them.

To mitigate this, Workato uses long polling with a wait timeout of 5 seconds. This means when Workato attempts to retrieve messages in triggers or actions, we will wait up to 5 seconds for messages to arrive at sampled servers. You may configure this timeout for actions using the input Wait time (seconds).

# Visibility timeout

Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consumers from receiving and processing the message.

The Workato connector defaults to 12 hours instead of SQS's 30 second default to minimise the possibility of jobs having the same messages. You may configure this timeout using the Visibility Timeout input field.

# Dead letter queues

Messages in an SQS queue have to be deleted to be considered processed. In cases where messages have been left in a queue for a set amount of time (configurable in SQS) or retrieved too many times, they are then stored in a dead-letter queue for debugging.

Workato does not actively support messages stored in dead-letter queues but recommends you set alarms to be notified on such messages being stored there. Learn more. (opens new window)

# Troubleshooting

# Duplicate messages are observed across jobs

In some cases, your jobs may be processed too slowly and messages that were retrieved by Workato have passed their visibility timeout and are eligible to be retrieved again. In such cases, you may see duplicate messages across different jobs.

Our recommendation is to increase the visibility timeout or delete messages as they are retrieved.