Redis - Publish Subscribe

Redis - Publish/Subscribe and Streams

Redis provides powerful messaging paradigms. The original ‘fire-and-forget’ model is Pub/Sub, while the newer, more robust model is Redis Streams, which provides a persistent, append-only log.

Classic Pub/Sub

Redis Pub/Sub is a messaging system where senders (Publishers) send messages to named ‘channels’, without knowledge of what, if any, subscribers there are. Receivers (Subscribers) express interest in one or more channels and receive messages only when they are connected and subscribed.

Key Characteristics of Pub/Sub

• Fire-and-Forget: If no subscribers are listening on a channel when a message is published, the message is lost forever.
• No History: Subscribers do not see messages that were published before they subscribed.
• Decoupled: Publishers and subscribers are completely decoupled and unaware of each other.

Example with redis-cli

Imagine two separate client sessions connected to the same Redis server.

-- Client A (Subscriber) --

redis> SUBSCRIBE live-updates
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "live-updates"
3) (integer) 1

-- At this point, Client A is blocked, waiting for messages --

-- Client B (Publisher) --

redis> PUBLISH live-updates "System rebooting in 5 minutes!"
(integer) 1

redis> PUBLISH live-updates "Reboot complete. All services are back online."
(integer) 1

-- Client A's console now shows the received messages --

1) "message"
2) "live-updates"
3) "System rebooting in 5 minutes!"

1) "message"
2) "live-updates"
3) "Reboot complete. All services are back online."

A More Powerful Alternative: Redis Streams

Introduced in Redis 5.0, Streams are a persistent, append-only log data structure. They provide a much more robust messaging system suitable for use cases like message queues, event sourcing, and durable notifications.

Key Characteristics of Streams

• Persistence: Messages are stored in the stream on the Redis server until explicitly deleted. Consumers can read messages from any point in history.
• Consumer Groups: Allows multiple clients to work together to consume messages from a single stream. Redis ensures each message is delivered to only one consumer within the group.
• Message Acknowledgment: Consumers must explicitly acknowledge (XACK) that they have processed a message. This allows for ‘at-least-once’ delivery and recovery from consumer failures.
• Blocking Reads: Similar to BRPOP for lists, clients can block until new messages are available in a stream.

Stream Example with redis-cli

-- Producer: Add a message to the 'orders' stream --
# The '*' tells Redis to auto-generate a unique ID (timestamp-sequence)

redis> XADD orders * user_id 123 product_id 987 quantity 2
"1672531200000-0"

-- Consumer: Read all messages from the beginning of the stream --
# The '0-0' is a special ID meaning 'the very first entry'

redis> XREAD COUNT 100 STREAMS orders 0-0
1) 1) "orders"
   2) 1) 1) "1672531200000-0"
         2) 1) "user_id"
            2) "123"
            3) "product_id"
            4) "987"
            5) "quantity"
            6) "2"

Pub/Sub vs. Streams: Which to Use?

• Use Pub/Sub for: Transient, real-time notifications where message loss is acceptable. Examples: live chat room notifications, notifying systems of a cache invalidation.
• Use Streams for: Durable, reliable messaging where every message must be processed. Examples: processing order submissions, job queues, event logs for microservices.

Relevant Commands

Pub/Sub Commands

Command	Description
SUBSCRIBE channel [channel …]	Listens for messages published to the given channels.
PUBLISH channel message	Posts a message to a channel.
PSUBSCRIBE pattern [pattern …]	Subscribes to channels matching glob-style patterns.
UNSUBSCRIBE [channel …]	Stops listening for messages on the given channels.

Core Stream Commands

Command	Description
XADD key ID field string [field string …]	Appends a new entry to a stream.
XREAD [COUNT count] [BLOCK ms] STREAMS key [key …] ID [ID …]	Reads one or more entries from one or more streams, optionally blocking.
XGROUP CREATE key groupname ID [MKSTREAM]	Creates a new consumer group associated with a stream.
XREADGROUP GROUP group consumer [COUNT count] [BLOCK ms] STREAMS key [key …] ID [ID …]	Reads from a stream as part of a consumer group.
XACK key group ID [ID …]	Acknowledges that one or more messages have been successfully processed.