Pub/Sub and Lightweight Queues

Introduction

Not every messaging problem needs Streams. Redis provides two simpler primitives for moving data between processes: Pub/Sub for real-time broadcasts and Lists for FIFO task queues. This chapter explains how both work, when to choose them over Streams, and the trade-offs you accept for their simplicity.

Prerequisites

A running Redis server and comfort with redis-cli
Familiarity with Lists (see Basic Data Types)
Understanding of the terms "producer" and "consumer"

Pub/Sub

Pub/Sub (publish/subscribe) is a fire-and-forget messaging pattern. A publisher sends a message to a channel. Redis forwards that message to every client currently subscribed to that channel. There is no persistence, no acknowledgment, and no memory of messages after delivery.

plaintext

Publisher ──→ Redis ──→ Subscriber A
                 └────→ Subscriber B
                 └────→ Subscriber C (offline: message lost)

Publishing and Subscribing

Open three terminal windows. In two of them, subscribe to a channel:

bash

# Terminal 1
redis-cli
SUBSCRIBE news:tech
 
# Terminal 2
redis-cli
SUBSCRIBE news:tech

In the third terminal, publish a message:

bash

# Terminal 3
redis-cli
PUBLISH news:tech "Redis 8 released"

Both subscribers immediately receive:

plaintext

1) "message"
2) "news:tech"
3) "Redis 8 released"

Pattern Subscriptions

PSUBSCRIBE lets you subscribe to multiple channels using glob-style patterns:

bash

# Subscribe to all channels starting with "news:"
PSUBSCRIBE news:*
 
# This subscriber now receives messages from news:tech, news:sports, etc.

Unsubscribing

bash

# Leave a specific channel
UNSUBSCRIBE news:tech
 
# Leave all pattern subscriptions
PUNSUBSCRIBE

In practice, subscribers usually disconnect rather than explicitly unsubscribe.

Warning

No Persistence, No Guarantees

If a subscriber is offline when a message is published, that message is lost forever. Pub/Sub does not buffer messages. If you need durability or replay, use Redis Streams instead.

When to Use Pub/Sub

Use Case	Why Pub/Sub Fits
Real-time notifications	Instant broadcast to all connected clients
Live dashboards	Push metric updates to browsers
Cache invalidation	Notify all app servers to drop a cached key
Presence indicators	Tell users when someone joins or leaves a chat room

Lists as Queues

Before Streams, Redis Lists were the standard way to build simple job queues. A producer pushes tasks to one end; a consumer blocks-waits and pops from the other.

Basic FIFO Queue

bash

# Producer: push jobs to the left
LPUSH queue:emails "welcome:user@example.com"
LPUSH queue:emails "invoice:1234"
LPUSH queue:emails "reminder:event@example.com"
 
# Consumer: block until a job is available, then pop from the right
BRPOP queue:emails 30
# 1) "queue:emails"
# 2) "welcome:user@example.com"

BRPOP returns (nil) if the timeout expires without a job. A typical worker loops forever:

python

# Pseudocode for a List-based worker
while True:
    result = redis.brpop("queue:emails", timeout=30)
    if result is None:
        continue  # No work, loop again
    queue_name, job_data = result
    process(job_data)

Reliable Queue with RPOPLPUSH

A naive BRPOP loses jobs if the consumer crashes after popping but before finishing processing. The Reliable Queue pattern uses BRPOPLPUSH (or its blocking variant) to move a job to a temporary "in-progress" list first.

bash

# Atomically pop from the queue and push to an in-progress list
BRPOPLPUSH queue:emails queue:emails:processing 30
 
# After processing, remove from the in-progress list
LREM queue:emails:processing 0 "welcome:user@example.com"

If the consumer crashes, the job remains in queue:emails:processing. Another worker can scan that list periodically and requeue stale jobs.

Tip

Consider Streams for New Projects

The List queue pattern works, but it requires manual handling of acknowledgments, retries, and multiple consumers. Redis Streams (see Streams) solve these problems natively. Use Lists only when you need the absolute simplest queue or are maintaining legacy code.

Pub/Sub vs List Queues vs Streams

Feature	Pub/Sub	List Queue	Streams
Persistence	No	Yes (until consumed)	Yes (until deleted)
Multiple consumers	Broadcast to all	One job per consumer	Consumer groups
Acknowledgment	No	Manual (with RPOPLPUSH)	Built-in (`XACK`)
Replay / history	No	No	Yes
Complexity	Lowest	Low	Medium

Choose Pub/Sub when every subscriber must receive the message and it is acceptable to drop messages for offline clients.
Choose List queues for simple worker pools with at-most-once or at-least-once delivery you manage yourself.
Choose Streams when you need persistence, competing consumers, delivery tracking, and the ability to re-read the event log.

FAQ

Can Pub/Sub channels scale to thousands of subscribers?

Yes. Redis pushes the same message to every subscriber in a loop. With very high subscriber counts or very large messages, CPU and network bandwidth on the Redis server become the bottleneck. For massive fan-out, consider external message brokers or sharding across multiple Redis instances.

Does Pub/Sub work on Redis Cluster?

Yes, but with a caveat: a client connected to one node only receives messages published to that same node. To receive all cluster-wide messages, you must subscribe on every master node, or use a client library that handles this automatically. Many developers prefer a dedicated single-node Redis instance for Pub/Sub to avoid this complexity.

How do I monitor Pub/Sub channels?

bash

# List active channels (with subscriber counts)
PUBSUB CHANNELS
PUBSUB NUMSUB news:tech

Can I set an expiration on a List used as a queue?

Yes. EXPIRE queue:emails 3600 deletes the entire List after one hour. You cannot expire individual List elements natively; for per-message TTL, use a Sorted Set with timestamps or a Stream.

What is the maximum message size in Pub/Sub?

Limited by the general Redis String limit (512 MB). In practice, keep Pub/Sub messages small (kilobytes) to avoid blocking the event loop while serializing and transmitting to many subscribers.