SQS vs SNS: Core Differences and When to Use Each AWS Messaging Service
You have an order placed event and three downstream systems — inventory, billing, and notifications — all need to react to it. Do you put the message in a queue and let each system poll it, or do you push it to all three at once? Getting this wrong means either building a polling nightmare or discovering that SQS only delivers each message to one consumer. This is the SQS vs SNS decision, and it comes up constantly in event-driven architecture design.
TL;DR: SQS vs SNS at a Glance
If you need one worker to process each message exactly once, use SQS. If you need one event to reach multiple independent subscribers simultaneously, use SNS. The two services solve fundamentally different problems and are frequently combined — SNS fans out, SQS absorbs the load per subscriber.
| Dimension | SQS | SNS |
|---|---|---|
| Delivery model | Pull (consumers poll) | Push (SNS pushes to subscribers) |
| Message consumers | One consumer per message | All subscribers receive every message |
| Persistence | Messages retained up to 14 days | No persistence — deliver or discard |
| Ordering | FIFO queues available | No ordering guarantee (standard topics) |
| Filtering | Not applicable at queue level | Subscription filter policies supported |
| Primary use case | Work queue, task decoupling | Fan-out, pub/sub broadcast |
How SQS and SNS Work — The Mechanism Behind the Difference
SQS is a distributed message queue. Producers write messages to a queue; consumers poll the queue, receive a batch, process it, and then explicitly delete the messages. Until a consumer deletes a message, it remains in the queue and becomes visible again after the visibility timeout expires. This design guarantees that exactly one consumer processes each message during any given visibility window — which is the entire point of a work queue.
SNS is a pub/sub notification service. A producer publishes a message to an SNS topic. SNS immediately pushes that message to every registered subscriber — Lambda functions, SQS queues, HTTP endpoints, email addresses, SMS, and others. There is no polling. There is no retention. If a subscriber is unavailable and has no dead-letter queue configured, the message is gone after SNS exhausts its retry attempts.
The architectural implication is significant: SQS serializes work across consumers, SNS multiplies delivery across subscribers. These are opposite behaviors.
(receives message)"] Q -.->|"hidden during
visibility timeout"| C2["Consumer B
(waits)"] end subgraph SNS_Model["SNS: Push Model"] P2["Producer"] --> T["SNS Topic"] T --> S1["Subscriber A
(Lambda)"] T --> S2["Subscriber B
(SQS Queue)"] T --> S3["Subscriber C
(HTTP Endpoint)"] end
- SQS pull model: The producer writes to the queue. Multiple consumers compete for messages — each message goes to exactly one consumer. The consumer deletes the message after successful processing.
- SNS push model: The producer publishes once to the topic. SNS immediately pushes the same message to all subscribers in parallel. No consumer competes with another.
- Visibility timeout: In SQS, a message in-flight is hidden from other consumers. If processing fails and the timeout expires, the message reappears for retry. SNS has no equivalent — retries are per-subscriber and limited.
When to Use SQS
Use SQS when you have a unit of work that should be processed once by one worker. The canonical examples: image resizing jobs, order fulfillment tasks, batch data processing, any background job that should not run twice.
SQS Standard queues offer at-least-once delivery with best-effort ordering. SQS FIFO queues provide exactly-once processing and strict ordering within a message group, at the cost of lower throughput. Choose FIFO when duplicate processing would cause data corruption — financial transactions, inventory adjustments.
SQS also gives you natural backpressure. If your consumers fall behind, messages accumulate in the queue rather than being dropped. You can scale consumers horizontally based on the ApproximateNumberOfMessagesVisible CloudWatch metric, which integrates directly with Application Auto Scaling.
The visibility timeout is the operational lever most engineers underestimate. Set it shorter than your actual processing time and you get duplicate deliveries. Set it too long and a crashed consumer holds a message invisible for minutes before it reappears. Match it to your p95 processing latency with some headroom.
When to Use SNS
Use SNS when a single event needs to trigger multiple independent reactions. The producer should not know or care who is listening. This is the pub/sub contract.
Classic scenarios: a new user registration event that simultaneously triggers a welcome email, a CRM record creation, and an analytics event. A payment completed event that fans out to fulfillment, accounting, and fraud review. In each case, the producer publishes once and walks away.
SNS subscription filter policies let individual subscribers declare which messages they want, based on message attributes. This means you can have a single SNS topic for all order events and let the fulfillment queue subscribe only to order_status: confirmed while the refund queue subscribes only to order_status: refunded. The filtering happens at the SNS layer — the producer publishes everything to one topic.
One thing SNS does not give you: durability. If your Lambda subscriber throws an unhandled error and you have not configured a dead-letter queue on the SNS subscription or the Lambda function, that delivery attempt fails and the message does not come back. Plan for this explicitly.
The SNS + SQS Fan-Out Pattern — the Most Common Production Combination
In practice, the most robust architecture combines both services. SNS handles fan-out; SQS absorbs the load per subscriber and provides durability, backpressure, and retry semantics that SNS alone cannot offer.
(publishes once)"] --> Topic["SNS Topic"] Topic --> QA["SQS Queue A
(Fulfillment)"] Topic --> QB["SQS Queue B
(Billing)"] Topic --> QC["SQS Queue C
(Notifications)"] QA --> CA1["Worker"] & CA2["Worker"] QB --> CB1["Worker"] QC --> CC1["Worker"] QA --> DLQA["DLQ A"] QB --> DLQB["DLQ B"] QC --> DLQC["DLQ C"]
- Producer publishes once to the SNS topic. It has no knowledge of downstream consumers.
- SNS delivers to multiple SQS queues — one per logical consumer group. Each queue is an independent subscriber.
- Each SQS queue buffers messages for its consumer fleet. Consumers scale independently based on queue depth.
- Dead-letter queues on each SQS queue catch messages that exhaust retry attempts, enabling forensic analysis without data loss.
This pattern is sometimes called SNS fan-out to SQS. It is the standard answer when you need both broadcast delivery and reliable per-consumer processing. The SQS queues decouple the consumers from SNS's push timing and provide the durability that SNS topics do not.
Think of SNS as a PA system announcement — everyone in the room hears it simultaneously. SQS is a ticket queue at a service counter — each ticket goes to exactly one agent. The fan-out pattern is a PA announcement that prints a ticket for each department simultaneously.
A Real Failure Pattern: Misrouting Work Through SNS Alone
The symptom: duplicate order fulfillments appearing in production. The initial diagnosis pointed at the fulfillment service itself — a suspected idempotency bug. Engineers spent two days reviewing the deduplication logic before someone pulled the SNS subscription list.
The actual cause: the fulfillment service had been deployed as two instances, both subscribed directly to the SNS topic as HTTP endpoints. SNS delivered to both. Every order event triggered two fulfillment attempts — by design, from SNS's perspective. The service was not buggy. The architecture was wrong.
The fix was straightforward: replace the two direct HTTP subscriptions with a single SQS queue subscription, and let both fulfillment instances compete for messages from that queue. One message, one fulfillment, regardless of how many instances are running.
This is the ordering dependency that breaks common assumptions: SNS does not know that two subscribers represent the same logical consumer. It treats them as independent subscribers and delivers to all of them. If you want competing consumers on a single logical stream, that is SQS's job, not SNS's.
IAM Permissions for SQS and SNS Operations
Both services require explicit IAM permissions. For the SNS-to-SQS fan-out pattern, the SQS queue must have a resource-based policy allowing SNS to send messages to it — IAM identity policies on the SNS service role alone are insufficient for cross-service delivery.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "sqs:SendMessage",
"Resource": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:sns:us-east-1:123456789012:MyTopic"
}
}
}
]
}
The aws:SourceArn condition restricts delivery to a specific SNS topic, preventing other topics from writing to your queue if the queue policy is ever broadened accidentally.
For producers publishing to SNS, the minimum required permission is sns:Publish on the topic ARN. For consumers polling SQS, the minimum set is sqs:ReceiveMessage, sqs:DeleteMessage, and sqs:GetQueueAttributes.
Key CLI Commands for SQS and SNS
Creating a topic, a queue, and wiring them together via subscription:
# Create an SNS topic
aws sns create-topic \
--name MyOrderTopic \
--region us-east-1
# Create an SQS queue
aws sqs create-queue \
--queue-name MyFulfillmentQueue \
--region us-east-1
# Subscribe the SQS queue to the SNS topic
aws sns subscribe \
--topic-arn arn:aws:sns:us-east-1:123456789012:MyOrderTopic \
--protocol sqs \
--notification-endpoint arn:aws:sqs:us-east-1:123456789012:MyFulfillmentQueue \
--region us-east-1
# Publish a test message to the SNS topic
aws sns publish \
--topic-arn arn:aws:sns:us-east-1:123456789012:MyOrderTopic \
--message '{"order_id": "abc-123", "order_status": "confirmed"}' \
--message-attributes '{"order_status": {"DataType": "String", "StringValue": "confirmed"}}' \
--region us-east-1
# Poll the SQS queue to verify delivery
aws sqs receive-message \
--queue-url https://sqs.us-east-1.amazonaws.com/123456789012/MyFulfillmentQueue \
--max-number-of-messages 1 \
--region us-east-1
Decision Guide: SQS vs SNS vs Both
systems need this event?"} Q1 -->|"Yes"| SNS["Use SNS Topic
(fan-out)"] Q1 -->|"No"| Q2{"Ordering or
exactly-once required?"} SNS --> Q3{"Each subscriber needs
durable retry?"} Q3 -->|"Yes"| Combo["SNS + SQS per subscriber
(fan-out pattern)"] Q3 -->|"No"| SNSOnly["SNS with direct
Lambda or HTTP subscribers"] Q2 -->|"Yes"| FIFO["SQS FIFO Queue"] Q2 -->|"No"| Standard["SQS Standard Queue"]
- Start by asking whether multiple independent systems need to react to the same event.
- If yes, SNS is the fan-out layer. Then ask whether each subscriber needs durable, retriable delivery — if yes, attach an SQS queue per subscriber.
- If only one system processes each message, go directly to SQS. Choose FIFO if ordering or exactly-once semantics are required.
Wrap-Up: SQS vs SNS in Production Systems
The SQS vs SNS distinction is not a preference — it reflects two fundamentally different delivery contracts. SQS serializes work to one consumer per message with durable retention. SNS broadcasts to all subscribers immediately with no persistence. Most production event-driven architectures use both: SNS for fan-out at the entry point, SQS for reliable per-consumer processing downstream.
For deeper reading, the AWS documentation covers SQS Developer Guide and the SNS Developer Guide in full. The fan-out pattern is documented under Using Amazon SQS as a subscriber for Amazon SNS.
Glossary
| Term | Definition |
|---|---|
| Visibility Timeout | The period during which SQS hides a received message from other consumers. If not deleted within this window, the message reappears in the queue. |
| Fan-out | The pattern of delivering one message to multiple independent subscribers simultaneously, typically via SNS. |
| Dead-Letter Queue (DLQ) | A secondary SQS queue that receives messages which could not be successfully processed after a configured number of attempts. |
| Subscription Filter Policy | An SNS feature that allows a subscriber to receive only messages matching specified attribute conditions, reducing unnecessary delivery. |
| FIFO Queue | An SQS queue type that guarantees message ordering within a message group and exactly-once processing, at reduced throughput compared to Standard queues. |
Comments
Post a Comment