Real-Time Distributed Chat System

Role: Personal project · Year: 2024 - 2025 · Status: shipped

tldr: A chat system built to study what actually breaks when WebSocket traffic has to live across multiple servers. Goroutines handle connections, Redis pub/sub fans out messages between nodes, a small custom load balancer sits in front. The interesting part is not the chat; it’s what stops working when you go from one node to two.

Uses a Go backend, React frontend, Redis Pub/Sub for cross-node fanout, and Docker containerization.

Executive summary

• Project Role: Creator / Sole Developer (Personal Project)
• Tech Stack: Go, React, WebSocket, Redis Pub/Sub, Docker
• Challenge: Horizontal scaling of stateful WebSocket connections across multiple nodes without losing message order or connection tracking.

The problem

Traditional real-time applications keep connection state in memory. While simple on a single node, scaling this architecture horizontally introduces a core problem: state fragmentation. A user connected to Server A cannot send a message directly to a user on Server B because Server A has no knowledge of Server B’s active connection handles. To build a robust, scalable system, connection management must be decoupled from the message distribution layer.

Constraints

• Commodity Hardware Limits: The system must run within strict memory bounds per node, precluding resource-heavy framework layers.
• WebSocket State Constraints: Active TCP connections are stateful and persistent; they cannot be shared or moved across physical servers without disconnects.
• Message Delivery Reliability: Messages must be delivered to target clients exactly once, avoiding duplicate writes or message loss due to node restarts.

Technical decisions

• Go for Backend Services: Selected Go for its light concurrency model. Goroutines allow handling thousands of concurrent connections with low memory footprint, avoiding callback/event loops.
• Redis Pub/Sub for Messaging Bus: Used Redis Pub/Sub for lightweight cross-instance coordination. Rather than synchronizing global client lists across all servers, Redis acts as a stateless broker. When a message is sent to Server A, it is broadcast to the channel, and Server B reads it to push down to its local clients.
• Docker for Orchestration: Packaged all services into lightweight containers to reliably test scaling from one node to multiple instances locally.

Trade-offs and results

• Decoupled Messaging vs. Instantaneous Consistency: Chose eventual consistency via pub/sub instead of distributed locking mechanisms. While a small latency offset exists during high throughput, connection management remains extremely fast and stable.
• Results: Successfully verified horizontal scaling of stateful connections with zero memory leaks. Goroutine footprints remained below 10KB per idle socket, allowing high concurrent throughput on standard instances.