我曾被要求评估RabbitMQ而不是Kafka,但发现很难找到一个消息队列比Kafka更适合的情况。有人知道在哪些用例中消息队列在吞吐量、持久性、延迟或易用性方面更适合吗?
当前回答
从技术上讲,与Rabbit MQ提供的特性集相比,Kafka提供了一个巨大的超特性集。
如果问题是
Rabbit MQ技术上比Kafka更好吗?
那么答案是
No.
但是,如果问题是
从业务角度看Rabbit MQ比Kafka好吗?
那么,答案是
在某些商业场景中,可能是“Yes”
从业务角度来看,Rabbit MQ可以比Kafka更好,原因如下:
Maintenance of legacy applications that depend on Rabbit MQ Staff training cost and steep learning curve required for implementing Kafka Infrastructure cost for Kafka is higher than that for Rabbit MQ. Troubleshooting problems in Kafka implementation is difficult when compared to that in Rabbit MQ implementation. A Rabbit MQ Developer can easily maintain and support applications that use Rabbit MQ. The same is not true with Kafka. Experience with just Kafka development is not sufficient to maintain and support applications that use Kafka. The support personnel require other skills like zoo-keeper, networking, disk storage too.
其他回答
简短的回答是“消息确认”。RabbitMQ可以配置为需要消息确认。如果接收方失败,消息将返回队列,另一个接收方可以再次尝试。虽然你可以用自己的代码在Kafka中完成这个任务,但它可以在RabbitMQ中开箱即用。
根据我的经验,如果你有一个需要查询信息流的应用程序,Kafka和KSql是你最好的选择。如果你想要一个排队系统,你最好使用RabbitMQ。
我知道有点晚了,也许你已经间接地说过了,但是,Kafka根本不是一个队列,它是一个日志(就像上面有人说的,基于民意调查)。
简单来说,当你更喜欢RabbitMQ(或任何队列技术)而不是Kafka时,最明显的用例是:
You have multiple consumers consuming from a queue and whenever there is a new message in the queue and an available consumer, you want this message to be processed. If you look closely at how Kafka works, you'll notice it does not know how to do that, because of partition scaling, you'll have a consumer dedicated to a partition and you'll get into starvation issue. Issue that is easily avoided by using simple queue techno. You can think of using a thread that will dispatch the different messages from same partition, but again, Kafka does not have any selective acknowledgment mechanisms.
你能做的最多的就是像那些家伙一样,试着把Kafka转换成一个队列: https://github.com/softwaremill/kmq
雅尼克
Scaling both is hard in a distributed fault tolerant way but I'd make a case that it's much harder at massive scale with RabbitMQ. It's not trivial to understand Shovel, Federation, Mirrored Msg Queues, ACK, Mem issues, Fault tollerance etc. Not to say you won't also have specific issues with Zookeeper etc on Kafka but there are less moving parts to manage. That said, you get a Polyglot exchange with RMQ which you don't with Kafka. If you want streaming, use Kafka. If you want simple IoT or similar high volume packet delivery, use Kafka. It's about smart consumers. If you want msg flexibility and higher reliability with higher costs and possibly some complexity, use RMQ.
你们忘记的一个关键区别是RabbitMQ是基于推的消息系统,而Kafka是基于拉的消息系统。这在消息传递系统必须满足具有不同处理能力的不同类型的消费者的场景中非常重要。使用基于Pull的系统,消费者可以根据自己的能力消费,而推送系统将推送消息,而不管消费者的状态如何,从而将消费者置于高风险之中。
我知道这是一个老问题了,但是在处理数据编校时RabbitMQ可能是一个更好的选择。
在RabbitMQ中,默认情况下,一旦消息被消费,它就会被删除。在Kafka中,默认情况下,消息保存一周。通常将这个时间设置为更长的时间,甚至永远不删除它们。
虽然这两个产品都可以配置为保留(或不保留)消息,但如果CCPA或GDPR合规性是一个问题,我会选择RabbitMQ。