Kafka vs RabbitMQ: Logs, Queues, Replay and When to Choose Each

The short answer

Kafka is a distributed log; RabbitMQ is a message broker with queues. In Kafka, producers append events to partitioned topics, the events stay for a retention period you set, and each consumer group tracks its own position, so the same data can be read many times and replayed. In RabbitMQ, publishers send to an exchange, bindings route a copy into queues, and a message leaves the queue once a consumer acknowledges it. Choose Kafka when many independent systems need the same event history, when you need replay, or when the data feeds stream processing. Choose RabbitMQ when you are distributing work between services, need flexible routing, or want standard protocols such as AMQP, MQTT and STOMP. The old split has softened: RabbitMQ has had a replicated log type, streams, since 3.9, and Kafka 4.2 made share groups, its queue-style consumption, production-ready.

Kafka vs RabbitMQ at a glance

How Kafka handles a message

Kafka's own documentation calls it an event streaming platform. Events are written to topics, and each topic is split into partitions spread across brokers. Events with the same key land in the same partition, and Kafka guarantees that a consumer reads a partition in exactly the order it was written. Events are not deleted when they are read. Each topic keeps data for a configured time or size, and log compaction can keep the latest value for every key indefinitely.

Consumers pull from brokers and record an offset. A consumer group shares a topic's partitions between its members, so a group can never use more active consumers than there are partitions. Because the log stays put, a consumer can rewind to an earlier offset and re-process, which is how teams recover from a bug in consumer code. Kafka adds transactions for exactly-once processing between Kafka topics, Kafka Connect for moving data in and out, and Kafka Streams for processing inside your application.

How RabbitMQ handles a message

RabbitMQ separates routing from storage. A publisher sends a message to an exchange. Bindings on that exchange decide which queues or streams receive a copy: a fanout exchange copies to every bound queue, a topic exchange matches routing-key patterns, and a headers exchange matches on message headers. Consumers register with a queue and the broker delivers to them, with a prefetch setting that caps how many unacknowledged messages each consumer holds.

The queue type is a real choice. Quorum queues replicate through Raft and are the choice where data safety and availability matter. Classic queues are local to one node; their mirroring was deprecated in 2021 and removed in RabbitMQ 4.0, which is the main thing to plan for when upgrading from 3.x. Streams are an append-only, replicated log with non-destructive reads, offset positioning and retention by size or age. Super streams, added in 3.11, partition a stream for ordered, parallel consumption.

The overlap: RabbitMQ streams and Kafka share groups

"Kafka for streaming, RabbitMQ for queues" is now only half true. RabbitMQ streams give RabbitMQ a durable log with replay. On the Kafka side, Queues for Kafka (KIP-932) was a preview in 4.1 and is production-ready in 4.2. Share groups let consumers cooperatively read records, outnumber the partitions they read from, acknowledge records one at a time and count delivery attempts, so a poison message can be handled automatically. Kafka's upgrade notes recommend share groups when records are processed one at a time rather than as an ordered stream.

So the question is no longer which product can do the job, but which model is the centre of gravity. RabbitMQ's queues are queues and its streams are a log. Kafka's share groups are a consumption mode on top of a log. If most of your traffic is work distribution with retries, RabbitMQ's queues are the more natural fit. If most of it is event history shared across teams, Kafka is.

Ordering, replay and delivery guarantees

• Ordering. Kafka orders within a partition, so choose the key carefully. RabbitMQ preserves order within a single queue with a single consumer; competing consumers on one queue process in parallel and ordering between them is not guaranteed. RabbitMQ's single active consumer and super streams cover the partitioned, ordered case.
• Replay. Native in Kafka and in RabbitMQ streams. Not possible from a RabbitMQ queue once a message is acknowledged.
• Delivery. Both support at-least-once delivery with acknowledgements or committed offsets. Kafka's transactions give exactly-once processing when both input and output are Kafka topics. Outside that boundary, in either system, make consumers idempotent.

Operating each one

Kafka runs in KRaft mode only from 4.0. A cluster still on ZooKeeper has to migrate to KRaft before it can move to 4.x, which our ZooKeeper to KRaft migration guide covers. Day-two work centres on partition counts, retention, rebalancing and client upgrades. Tiered storage can move completed segments to object storage, though not for compacted topics.

RabbitMQ runs on the Erlang/OTP runtime, and each RabbitMQ release supports specific Erlang versions, so the runtime moves with the broker. The work centres on queue types, memory and disk alarms, and moving from mirrored classic queues to quorum queues before or during a 4.x upgrade. Our guide to migrating mirrored queues to quorum queues walks through that change.

Which should you choose, Kafka or RabbitMQ?

• Choose Kafka if several teams consume the same events independently, you need replay or long retention, you feed stream processing or a data platform, or you want Kafka Connect's connector ecosystem.
• Choose RabbitMQ if services hand work to each other, you need routing by pattern or header, you want per-message acknowledgement and dead-lettering as the default behaviour, or clients speak AMQP, MQTT or STOMP.
• Run both when the estate genuinely has both shapes of traffic. A typical split is RabbitMQ for commands and tasks between services and Kafka for the event history that analytics and other teams read.
• Stay where you are if the only reason to move is that the other one is fashionable. Each now covers more of the other's ground than it used to.

Licensing and commercial support

Apache Kafka is Apache 2.0 under the Apache Software Foundation. Confluent sells a platform built around it with its own licences; our Apache Kafka vs Confluent post covers that line. RabbitMQ's server is MPL 2.0, and Broadcom holds the copyright and sells commercial editions and long-term support; see OSSeva vs Broadcom Tanzu RabbitMQ and the RabbitMQ licence explainer. For a wider view of who supports each, see our lists of Kafka support providers and RabbitMQ support providers.

Where OSSeva fits

OSSeva supports both, so the choice can follow the workload rather than the contract. OSSeva for Apache Kafka ships patched builds from 2.8 through 3.9, including 3.8 and 3.9 after upstream archived them, and covers 4.x in KRaft mode. OSSeva for RabbitMQ patches community RabbitMQ from 3.8 through 4.3, including 3.13 and 4.0 to 4.2 after community end of life, along with the Erlang/OTP runtime underneath. You keep the binaries you run today and change who supports them. Both sit under one contract with your databases, priced per cluster. Book a discovery call for a quote.

Frequently asked questions

Kafka vs RabbitMQ vs SQS: which should we use?

Amazon SQS is a hosted queue on AWS with standard queues (at-least-once delivery) and FIFO queues (exactly-once processing). It keeps a message for four days by default and fourteen at most, and there is no broker to run. Choose SQS for simple work queues that live entirely on AWS. Choose RabbitMQ when you need richer routing, standard protocols or the same broker across clouds and data centres. Choose Kafka when you need long retention, replay or many independent readers of the same events.

Kafka vs RabbitMQ vs Redis: when is Redis enough?

Redis Pub/Sub, and the same feature in Valkey, is at-most-once: a subscriber that is disconnected or fails simply misses the message. Redis and Valkey streams add an append-only log with consumer groups and are a reasonable choice for light event traffic inside an application that already runs Redis or Valkey. For durable, replicated messaging between many services, use RabbitMQ or Kafka. Our Valkey vs Redis post covers that choice.

Kafka vs RabbitMQ for microservices: which is better?

For request-style commands and background jobs between services, RabbitMQ is usually simpler: routing, acknowledgements, retries and dead-letter queues are built in. For event-driven designs where services publish facts that others build their own state from, Kafka fits better because the log is retained and replayable. Using both in one estate is a sound design.

Kafka vs RabbitMQ vs ActiveMQ: how does ActiveMQ compare?

ActiveMQ is a Java message broker built around JMS. It sits closer to RabbitMQ than to Kafka: queues and topics, acknowledgements, and protocols such as OpenWire, AMQP, STOMP and MQTT. Choose it when your applications use the JMS API. Our RabbitMQ vs ActiveMQ comparison covers the Classic and Artemis brokers.

Is Kafka faster than RabbitMQ?

There is no single answer. Kafka is designed for high throughput through sequential log writes and batched pulls; RabbitMQ pushes each message to a consumer as it arrives and tracks acknowledgement per message. Results depend on message size, durability settings, queue type and client configuration. Test your own workload on both rather than relying on published benchmarks.

Can RabbitMQ replace Kafka, or Kafka replace RabbitMQ?

Sometimes. RabbitMQ streams cover replay and fan-out to many readers, and Kafka share groups cover queue-style work distribution. Replacing one with the other still means changing clients, operations and monitoring, so do it only when the workload clearly fits the other model better.