Messaging With Kafka and RabbitMQ

Introduction

Message queues decouple services—order service publishes OrderCreated, email service consumes asynchronously. Spring Boot integrates Kafka and RabbitMQ via starters with minimal boilerplate. This chapter covers producer/consumer basics, acknowledgment, retries, and idempotency concepts for both brokers.

Prerequisites

• Transaction Management
• Docker for local broker (see Docker intro)

When to Pick Which

Learning either teaches async messaging patterns—many teams use both.

RabbitMQ Quick Start

docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

Management UI: http://localhost:15672 (guest/guest—change in prod).

Dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

Config:

spring:
  rabbitmq:
    host: localhost
    port: 5672
    username: guest
    password: guest

RabbitMQ Producer

Declare exchange/queue (config class):

RabbitMQ Consumer

@Component
public class OrderCreatedListener {
 
    private static final Logger log = LoggerFactory.getLogger(OrderCreatedListener.class);
 
    @RabbitListener(queues = "orders.created.queue")
    public void onOrderCreated(OrderEventPublisher.OrderCreatedEvent event) {
        log.info("Send email for orderId={}", event.orderId());
    }
}

Manual ack for reliability:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: manual

Kafka Quick Start

docker run -d --name kafka -p 9092:9092 \
  -e KAFKA_CFG_NODE_ID=0 \
  -e KAFKA_CFG_PROCESS_ROLES=controller,broker \
  -e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093 \
  -e KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT \
  -e KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@localhost:9093 \
  -e KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER \
  bitnami/kafka:3.7

Or use local docker-compose with ZooK-less KRaft—adjust for your image docs.

Dependency:

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

Config:

Kafka Producer

Kafka Consumer

@Component
public class KafkaOrderListener {
 
    @KafkaListener(topics = "order-events", groupId = "demo-group")
    public void listen(OrderEventPublisher.OrderCreatedEvent event) {
        // process idempotently
    }
}

Idempotency and Retries

Consumers may see duplicate messages—design handlers idempotent:

• Store processed message id in DB with unique constraint
• Use business key (orderId) "already handled" check

Retry with backoff via Spring @RetryableTopic (Kafka) or Rabbit RetryInterceptor.

Dead letter queue (DLQ) holds poison messages after max retries—investigate manually.

Outbox Pattern (Concept)

Publish after DB commit: write event to outbox table in same transaction; separate poller publishes to broker—avoids "DB committed but message lost".

FAQ

Message not consumed?

Wrong queue/topic name, consumer group offset past end, or deserialization failure—check logs.

JSON deserialize failed?

trusted.packages must include event class package.

Kafka ordering?

Only guaranteed within partition—key messages by orderId.

Transactional Kafka?

Advanced KafkaTransactionManager—exactly-once between DB and Kafka needs careful setup.

Local dev without Docker?

Testcontainers in integration tests spin real broker containers.

Rabbit vs JMS?

spring-boot-starter-amqp is Rabbit-native—different from generic JMS.