我有一个用例,将有数据流到来,我不能以相同的速度消费它,需要一个缓冲区。这可以使用SNS-SQS队列来解决。我后来才知道,Kinesis解决了同样的目的,所以有什么不同?为什么我应该喜欢(或不应该喜欢)运动?


当前回答

Kinesis解决了流数据典型的映射缩减场景中的映射部分问题。而SQS并不确定这一点。如果你有需要在一个键上聚合的流数据,kinesis可以确保该键的所有数据都到一个特定的分片,并且该分片可以在单个主机上使用,这使得在键上的聚合比SQS更容易

其他回答

对我来说,最大的优势是Kinesis是一个可重玩的队列,而SQS不是。因此,您可以有多个Kinesis的相同消息的消费者(或在不同时间的相同消费者),而在SQS中,一旦消息被ack,它就从队列中消失了。 因此,SQS更适合工作者队列。

这些技术的语义不同,因为它们是为支持不同的场景而设计的:

SNS/SQS:流中的项之间没有关联 运动:流中的项目相互关联

让我们通过例子来理解其中的区别。

Suppose we have a stream of orders, for each order we need to reserve some stock and schedule a delivery. Once this is complete, we can safely remove the item from the stream and start processing the next order. We are fully done with the previous order before we start the next one. Again, we have the same stream of orders, but now our goal is to group orders by destinations. Once we have, say, 10 orders to the same place, we want to deliver them together (delivery optimization). Now the story is different: when we get a new item from the stream, we cannot finish processing it; rather we "wait" for more items to come in order to meet our goal. Moreover, if the processor process crashes, we must "restore" the state (so no order will be lost).

一旦一个项目的处理不能与另一个项目的处理分离,我们就必须有运动语义,以便安全地处理所有的情况。

In very simple terms, and keeping costs out of the picture, the real intention of SNS-SQS are to make services loosely coupled. And this is only primary reason to use SQS where the order of the msgs are not so important and where you have more control of the messages. If you want a pattern of job queue using an SQS is again much better. Kinesis shouldn't be used be used in such cases because it is difficult to remove messages from kinesis because kinesis replays the whole batch on error. You can also use SQS as a dead letter queue for more control. With kinesis all these are possible but unheard of unless you are really critical of SQS.

如果你想要一个好的分区,那么SQS将不会有用。

Kinesis解决了流数据典型的映射缩减场景中的映射部分问题。而SQS并不确定这一点。如果你有需要在一个键上聚合的流数据,kinesis可以确保该键的所有数据都到一个特定的分片,并且该分片可以在单个主机上使用,这使得在键上的聚合比SQS更容易

Kinesis用例

日志和事件数据收集 实时分析 移动数据采集 “物联网”数据馈送

SQS用例

应用程序集成 解耦microservices 将任务分配给多个工作节点 将实时用户请求与密集的后台工作分离 批处理消息以供将来处理