Requirements

Functional Requirements:

• Post Comments in Real Time
• Users can send comments that should appear instantly to all participants in the same chat room.
• Subscribe to Live Comments via WebSockets
• Clients maintain persistent WebSocket connections to receive new comments without polling.
• Multiple Rooms / Streams
• The system supports many concurrent chat rooms (e.g., different live streams).
• Moderation Tools
• Moderators can delete comments, block users, or restrict posting. Deletions must propagate in real time.
• Load Historical Comments
• When a user joins a room or scrolls, the client should be able to fetch previously persisted comments.

Non-Functional Requirements:

• Scalability
• Must support millions of concurrent users and very high write throughput.
• Low Latency
• Real-time fan-out must be <100ms end-to-end.
• High Availability
• WebSocket servers, message queues, and databases should tolerate failures without interrupting the chat.
• Eventual Consistency (Acceptable)
• Small inconsistencies (e.g., a message shown live but lost before persistence) are acceptable for this product.

API Design

• WebSocket: SEND_COMMENT
  • { type: "send_comment", roomId, userId, content }

• WebSocket: MODERATION_ACTION
  • { type: "delete_comment", commentId }
  • { type: "ban_user", userId }

• GET /rooms/{roomId}/history?before=timestamp&limit=n

High-Level Design

1. WebSocket Gateway Cluster
2. • Maintains persistent connections.
   • Handles fan-out of new comments.
   • Routes users based on room ID (room-based affinity).
4. In-Memory Cache (Redis)
5. • Temporarily stores recent messages.
   • Used for quick fan-out and reducing DB reads.
   • AOF (Append-Only File) or replication can optionally reduce loss.
7. Message Queue (Kafka/RabbitMQ)
8. • Every incoming comment is pushed to the queue.
   • Ensures reliable, durable handoff.
   • Enables asynchronous persistence.
10. Comment Persistence Service
11. • Consumes from the message queue.
    • Writes comments to the database.
    • Handles moderation updates.
13. Database Layer (SQL chosen)
14. • Comments, Users, Rooms tables.
    • Strong consistency for historical data.
    • Scaling via table-level sharding by room ID.
16. Load Balancer
17. • Routes WebSocket connections.
    • Automatically fails users over to healthy WebSocket servers.
19. Client
20. • Maintains WebSocket connection.
    • Prefetches user metadata once (cached locally).
    • Applies incoming updates (new comments / deletions).

Detailed Component Design

WebSocket Server

• Sharded by Room ID, so all users of room X land on the same server.
• If a user joins multiple rooms → multiple WS connections → acceptable trade-off.
• If a server crashes:
• • Client auto-reconnects.
  • Load balancer reassigns to a new server.
  • Redis and queue minimize loss.

Message Queue (Kafka Recommended)

• Provides:
• • Replication
  • Durability
  • Replayability
  • Back-pressure handling
• Ensures the WebSocket server never blocks while persisting writes.

Database (SQL)

Chosen for:

• Clean relational modeling (users, rooms, comments).
• Strong consistency for history.
• Easy moderation queries.

Sharding Strategy:

• Shard Comments by room ID.
• Users stored separately on a different shard set.
• Avoid cross-shard joins by:
• • Preloading user metadata when joining a room.
  • Treating user profile changes as rare.

Caching Strategy

• When joining a room:
• • Client fetches user metadata for all users visible in chat.
• Real-time username changes → WebSocket event.
• Prevents cross-shard joins during history loads.

Moderation

• Moderator deletes comment:
• • A delete event is broadcast to clients over WebSocket.
  • Persistence service marks the comment deleted in DB.
• Clients remove comment from UI immediately.

Failure Handling

• WS Server Fail:
• • Reconnect clients → load balancer → new WS server.
• Redis Fail:
• • Only recent cache is lost; client already received messages.
  • DB + queue ensure long-term consistency.
• Queue Fail:
• • Kafka replication ensures minimal loss.
• Message Loss Window:
• • A comment broadcast may be shown but not yet persisted if server dies at the exact microsecond.
  • This is acceptable for real-time systems like chat.