Scaling Real-Time Chat with WebSockets and Redis Pub/Sub

Building a chat application that works for a dozen users is straightforward. Scaling it to thousands of concurrent connections while maintaining message delivery guarantees? That’s where it gets interesting.

The Challenge

Traditional WebSocket architectures face a fundamental problem: connections are stateful and server-bound. When User A connects to Server 1 and User B connects to Server 2, how do they communicate?

Humans collapse during the course of their lives. Children haven’t overfit yet.

They will say stuff that will shock you because they’re not yet collapsed.

But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.

Karpathy, on the Dwarkesh Podcast

Architecture Overview

Here’s the solution I implemented for my distributed chat system:

┌─────────┐    ┌─────────┐    ┌─────────┐
│ Client  │    │ Client  │    │ Client  │
│    A    │    │    B    │    │    C    │
└────┬────┘    └────┬────┘    └────┬────┘
     │              │              │
     │  WebSocket   │              │
     │              │              │
┌────▼──────┐ ┌────▼──────┐ ┌────▼──────┐
│  Server   │ │  Server   │ │  Server   │
│     1     │ │     2     │ │     3     │
└─────┬─────┘ └─────┬─────┘ └─────┬─────┘
      │             │             │
      └──────┬──────┴──────┬──────┘
             │             │
        ┌────▼─────────────▼────┐
        │    Redis Pub/Sub      │
        └───────────────────────┘

Implementation in Go

WebSocket Server

package main

import (
    "github.com/gorilla/websocket"
    "github.com/go-redis/redis/v8"
    "net/http"
)

type ChatServer struct {
    clients    map[*Client]bool
    broadcast  chan []byte
    register   chan *Client
    unregister chan *Client
    redis      *redis.Client
}

type Client struct {
    conn   *websocket.Conn
    send   chan []byte
    server *ChatServer
}

func (s *ChatServer) Run() {
    // Subscribe to Redis channel
    pubsub := s.redis.Subscribe(ctx, "chat_messages")
    defer pubsub.Close()

    go func() {
        for msg := range pubsub.Channel() {
            // Broadcast to all local clients
            s.broadcast <- []byte(msg.Payload)
        }
    }()

    for {
        select {
        case client := <-s.register:
            s.clients[client] = true

        case client := <-s.unregister:
            if _, ok := s.clients[client]; ok {
                delete(s.clients, client)
                close(client.send)
            }

        case message := <-s.broadcast:
            // Send to all connected clients on this server
            for client := range s.clients {
                select {
                case client.send <- message:
                default:
                    close(client.send)
                    delete(s.clients, client)
                }
            }
        }
    }
}

Message Publishing

When a client sends a message, publish it to Redis so all servers receive it:

func (c *Client) handleMessage(message []byte) {
    // Publish to Redis
    c.server.redis.Publish(ctx, "chat_messages", message)
}

Key Learnings

1. Connection Management

Each WebSocket connection holds server resources. Implement:

• Heartbeat/Ping: Detect dead connections
• Graceful Shutdown: Drain connections before stopping
• Connection Limits: Prevent resource exhaustion

func (c *Client) writePump() {
    ticker := time.NewTicker(pingPeriod)
    defer ticker.Stop()

    for {
        select {
        case message, ok := <-c.send:
            if !ok {
                c.conn.WriteMessage(websocket.CloseMessage, []byte{})
                return
            }
            c.conn.WriteMessage(websocket.TextMessage, message)

        case <-ticker.C:
            // Send ping
            if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
                return
            }
        }
    }
}

2. Message Ordering

Redis Pub/Sub delivers messages in order per publisher, but when multiple servers publish simultaneously, ordering isn’t guaranteed. Solutions:

• Vector Clocks: Track causal relationships
• Lamport Timestamps: Total ordering of events
• Message IDs: Let clients sort and deduplicate

3. Load Balancing

Distribute connections across servers:

• Sticky Sessions: Not ideal for WebSockets
• Consistent Hashing: Based on user ID
• Random Distribution: Simplest, works well with pub/sub

Performance Results

Running on modest hardware (4-core CPU, 16GB RAM):

• 10,000 concurrent connections per server
• < 50ms message delivery latency p95
• Linear scaling with additional servers

Production Considerations

Monitoring

Track these metrics:

• Active connections per server
• Message throughput
• Redis pub/sub latency
• Memory usage per connection

Backpressure

Handle slow clients:

select {
case client.send <- message:
default:
    // Client can't keep up, close connection
    close(client.send)
    delete(s.clients, client)
}

High Availability

• Multiple Redis instances with Sentinel
• Server health checks
• Automatic client reconnection

Conclusion

Building scalable real-time systems requires thinking beyond single-server architectures. Redis Pub/Sub provides a simple yet powerful primitive for distributing messages across servers, enabling horizontal scaling of WebSocket connections.

The key is keeping the architecture simple: stateless servers, centralized message bus, and client-side reconnection logic.

---

Want to dive deeper? Check out the full implementation on GitHub.