Requirements

Functional Requirements:

• support 1-to-1 chat (send/recieive)
• support group chat (create, manage, delete group and send/recieve)
• support for images/media (size limit of 25MBs)
• Message status: each message goes sent → delivered → read, tracked
• Read receipts: "Users can see when their message was delivered and read by each recipient" (optional)
• quick ordering convergance (in correct order)
• keep history of ~1 year

Non-Functional Requirements:

• 500M registered users total
• ~100M DAU active each day
• ~20M concurrent on average, ~100M peak concurrent (e.g., NYE at midnight when everyone's messaging)
• ~2 messages per user per minute on average
• Peak throughput ≈ 3.3M messages/sec writes (100M concurrent × 2 msg/min ÷ 60s) — plus the same order of read/fan-out traffic
• reliability
• scalability!
• fast message delivery (instant)

Capacity Estimation

500M registered users total - this data can sit on a PostgersqlDB

100M DAU active each day

Peak throughput ≈ 3.3M messages/sec writes (100M concurrent × 2 msg/min ÷ 60s) — plus the same order of read/fan-out traffic

TStorage Estimation (1-year retention)

• Text history: average user sends ~50 msgs/day (not peak) → 100M × 50 = 5B msgs/day × 0.1 KB = ~500 GB/day → ~180 TB/year → ~0.5–1 PB with replication + index overhead.
• Media: only ~5–10% of messages carry media → separate object storage + CDN; hot DB stores only the URL/metadata.
• Metadata (users + groups): 500M users + ~100M groups × ~1 KB ≈ ~1.5 TB → fits comfortably in a few relational DBs.

API Design

/api/user/:id/group [POST]

/api/user/:id/group/:groupId [DELETE]

/api/user/:id/message/:otherUserId [POST]

/api/user/:id/message/:groupId [POST]

/api/user/:id/media/:groupId [POST]

/api/user/:id/messages/:group:id [GET]

GroupId can be created for every 1on1 as well

Join chat / subscribe 

A chat system's defining API is the real-time channel — e.g., WebSocket: ws://server/ws?userId=...

High-Level Design

Authentication Service - to make sure that client can be supported to sending

SendingService - responsible for passing messages between users and users & groups. This is responsible for establishing real time channels (WebSockets) as well.

ArchiveService - responsible for moving older messages to historicMessagesDB

GroupsDB, MessagesDB, UsersDB, HistoricMessagesDB

MessagesQueue - storing data to be pushed for sending

MessagesWorker - job that pulls messages from the queue and forwards this to Sending Provider

LocalMessagesCache - Every message in a conversation gets a monotonic ID (per-conversation sequence number).

The client stores the last message ID it received locally.

On reconnect: client re-opens the WebSocket, then pulls the gap via REST — GET /messages?after=<last_id>.

Delivery is at-least-once; the client dedupes by message ID.

RateLimiter (API Gateway) - A rate-limiter rejects with a trivial check (a Redis counter). Letting abuse traverse the LB, routing layer, and land at SenderService means you've spent real CPU/network on traffic you're about to throw away.

"Messages land on a conversation topic once; all WS nodes subscribed to that topic consume and deliver in parallel."

Database Design

USERS (id, name, email, hashed_pwd, prefs)

GROUPS (is, name, owner_id, type: 1on1 vs group)

GROUP_MEMBERS (id, userid, role, joined_at, last read msssg)

MESSAGES (id, senderId, groupid, timestamp, text, medialinks)

read-receipt tracking: store last_read_msg_id on GROUP_MEMBERS — that gives per-recipient read state for free.

Shard Messages by sender_id with consistent hashing — a hot user's writes stay on one partition, and consistent hashing redistributes load evenly as nodes are added/removed.

Detailed Component Design

1. ArchiveService is running every day and making sure all data older than 1 year are accessible from the other table (moved to the other). nightly job moves messages older than 1 year to cold storage; reads fall back to it transparentl
2. WebSocket Gateway (stateless, horizontally scalable)

• Reconnect/resume: client stores last_message_id per conversation; on reconnect it re-subscribes and pulls GET /messages?after=<id>, deduping by msg_id — no gaps, no duplicates.
• Node change (deploy/failure): gateways are stateless; the connection registry in Redis rebinds the session to any healthy node, so LB rerouting never breaks a chat.

3. Pub/Sub bus (Kafka/Pulsar)

• Per-group ordering: topic partitioned by group_id (single partition per group) → messages in one chat arrive in order despite fan-out and retries.
• Dedupe on retry: messages carry a unique msg_id; producers/consumers are idempotent, so retries don't duplicate.

4. Scaling at peak (the 3.3M msg/s answer)

• Stateless gateway tier autoscales on connection count/CPU.
• Kafka partitions grow with consumer groups; consumer lag triggers scale-out.
• Cassandra absorbs write bursts via LSM append-only; add nodes, rebalance with consistent hashing.
• Redis registry sharded so 100M connections are tracked without a hotspot.