我什么时候使用SNS和SQS,为什么它们总是耦合在一起?


当前回答

以下是AWS上主要消息传递技术(SQS, SNS, +EventBridge)之间的主要区别。为了选择特定的AWS服务,我们应该了解该服务提供的功能以及与其他服务的比较。

下图总结了该服务之间的主要相似点和不同点。

其他回答

SNS和SQS之间有一些关键的区别:

SNS supports A2A and A2P communication, while SQS supports only A2A communication. SNS is a pub/sub system, while SQS is a queuing system. You'd typically use SNS to send the same message to multiple consumers via topics. In comparison, in most scenarios, each message in an SQS queue is processed by only one consumer. With SQS, messages are delivered through a long polling (pull) mechanism, while SNS uses a push mechanism to immediately deliver messages to subscribed endpoints. SNS is typically used for applications that need real time notifications, while SQS is more suited for message processing use cases. SNS does not persist messages - it delivers them to subscribers that are present, and then deletes them. In comparison, SQS can persist messages (from 1 minute to 14 days).

单独地,Amazon SQS和SNS用于不同的用例。但是,您可以在某些场景中将它们一起使用。

SNS是一个分布式发布-订阅系统。当信息由发布者发送到SNS时,就会推送给订阅者。

SQS is distributed queuing system. Messages are not pushed to receivers. Receivers have to poll or pull messages from SQS. Messages can't be received by multiple receivers at the same time. Any one receiver can receive a message, process and delete it. Other receivers do not receive the same message later. Polling inherently introduces some latency in message delivery in SQS unlike SNS where messages are immediately pushed to subscribers. SNS supports several end points such as email, SMS, HTTP end point and SQS. If you want unknown number and type of subscribers to receive messages, you need SNS.

你不必总是把SNS和SQS结合在一起。你可以让SNS发送消息到电子邮件,短信或HTTP端点除了SQS。将SNS与SQS耦合有很多优点。您可能不希望外部服务连接到您的主机(防火墙可能会阻止所有来自外部的传入连接到您的主机)。

你的终点可能会因为大量的消息而消失。电子邮件和短信可能不是你快速处理信息的选择。通过将SNS与SQS耦合,您可以按照自己的节奏接收消息。它允许客户端离线,容忍网络和主机故障。您还可以实现有保证的交付。如果将SNS配置为向HTTP端点或电子邮件或SMS发送消息,则多次发送消息失败可能导致消息被丢弃。

SQS主要用于解耦应用程序或集成应用程序。消息可以在SQS中存储短时间(最多14天)。SNS将多个消息副本分发给多个订阅者。例如,假设您希望将应用程序生成的数据复制到多个存储系统。您可以使用SNS并将此数据发送给多个订阅者,每个订阅者将其接收到的消息复制到不同的存储系统(S3、主机上的硬盘、数据库等)。

来自AWS文档:

Amazon SNS允许应用程序发送时间紧迫的消息到 多个订阅者通过“推送”机制,消除了需求 定期检查或“轮询”更新。 Amazon SQS是分布式应用程序使用的消息队列服务 通过轮询模型交换消息,并可用于 分离发送和接收组件——不需要每个组件 组件要同时可用。

扇出到Amazon SQS队列

AWS SNS是一个发布者订阅者网络,订阅者可以在其中订阅主题,并在发布者发布该主题时接收消息。

AWS SQS是一个队列服务,它将消息存储在队列中。SQS不能传递任何消息,需要一个外部服务(lambda、EC2等)来轮询SQS并从SQS获取消息。

由于多种原因,可以将SNS和SQS结合使用。

There may be different kinds of subscribers where some need the immediate delivery of messages, where some would require the message to persist, for later usage via polling. See this link. The "Fanout Pattern." This is for the asynchronous processing of messages. When a message is published to SNS, it can distribute it to multiple SQS queues in parallel. This can be great when loading thumbnails in an application in parallel, when images are being published. See this link. Persistent storage. When a service that is going to process a message is not reliable. In a case like this, if SNS pushes a notification to a Service, and that service is unavailable, then the notification will be lost. Therefore we can use SQS as a persistent storage and then process it afterwards.

以下是两者的对比:

实体类型

队列(类似于JMS) SNS: Topic (Pub/Sub系统)

信息消费

SQS:拉取机制——消费者从SQS中轮询和拉取消息 SNS:推送机制——SNS向消费者推送消息

用例

SQS:解耦两个应用程序并允许并行异步处理 SNS: Fanout -以多种方式处理相同的消息

持久性

SQS:如果没有消费者可用,消息将被持久化一段时间(可配置的)(最多两周),因此当消息添加到队列时,消费者不必处于空闲状态。 社交网络:没有持久性。在消息到达时出现的任何一个消费者都将获得消息并删除消息。如果没有可用的消费者,则在几次重试后消息将丢失。

消费者类型

SQS:所有消费者通常都是相同的,因此以完全相同的方式处理消息(每个消息由一个消费者处理一次,尽管在极少数情况下消息可能会被重新发送) SNS:消费者可能以不同的方式处理消息

示例应用程序

SQS: Jobs framework: The Jobs are submitted to SQS and the consumers at the other end can process the jobs asynchronously. If the job frequency increases, the number of consumers can simply be increased to achieve better throughput. SNS: Image processing. If someone uploads an image to S3 then watermark that image, create a thumbnail and also send a Thank You email. In that case S3 can publish notifications to an SNS topic with three consumers listening to it. The first one watermarks the image, the second one creates a thumbnail and the third one sends a Thank You email. All of them receive the same message (image URL) and do their processing in parallel.