Requirements

Functional Requirements:

• 1-1 messaging between users
• Group messaging
• Read receipts
• Notifications
• User can see Message history last ~1 year

Non-Functional Requirements:

• Low latency: <100ms delivery target
• High availability
• Highly scalable (billions of messages/day)
• Durability - no message loss
• Ordering of conversation messages

Capacity Estimation

• 1B users
• 100M DAU
• for each user ~50 messages/day
  • 50B Messages/day
  • ~60K messages/sec avg

Peak maybe 10x ~ 600k messages/sec

API Design

Chat history

Get /chats/{id}/messages

Chat list

Get /chats

Group APIs

Create group

POST /groups

Add member

POST /groups/{id}/members

Update group = disable/delete

PUT /groups/{id}

gRPC - for sending/delivering real time messages to group or single user

sendMessage(from, to, text)

Joining chat - via websockets. when user opens app a new websocket connection is created so user can recieve message (via chat Service using the active socket connection

1. API Gateway

• Authenticates users
• Rate limiting (user + group level)
• Routes requests to Chat Service / WebSocket Gateway

2. WebSocket Gateway

• Maintains persistent connections
• Forwards:
  • outgoing messages → Chat Service
  • read events → Chat Service
• Uses Presence Service to map:
• userId + gatewayId

3. Presence Service

Backed by Redis.

Stores:

• userId → set(deviceId, gatewayId)
• heartbeat-based online status

Used for:

• routing messages
• online/offline detection

4. Chat Service (System of Record)

Responsible for:

• assigning sequenceNumber
• persisting messages
• producing Kafka events

Key guarantee:

👉 Single source of truth for message ordering

5. Delivery Service

Consumes MessageCreated events:

• checks presence
• delivers via WebSocket
• updates delivery state

Stores lastRecievedSeq in redis after delivery attempt

👉 Best-effort real-time delivery only

6. Sync Service

Triggered on reconnect:

• fetches missed messages using:
• lastRecievedSeq
• ensures eventual consistency

7. Notification Service

Consumes Kafka events:

• sends push notifications only if user is offline

8. Group Service

Manages:

• group creation
• membership updates
• role management

9. Kafka (Event Backbone)

Handles:

• MessageCreated
• MessageRead
• Sync events

Ordering guarantee:

• 👉 conversationId used as partition key

We use a NoSQL store (e.g., Cassandra / DynamoDB-style model) optimized for:

• high write throughput (~50B messages/day)
• horizontal scalability
• ordered sequential reads per conversation

Data is modeled around access patterns, mainly:

• send message
• fetch chat history
• fetch conversation list
• track read state

Data Models

Message

• conversationId (partition key)
• sequenceNumber (ordering key)
• messageId
• senderId
• content (encrypted)
• createdAt

👉 Messages are stored append-only per conversation and strictly ordered by sequenceNumber

Conversation

• conversationId
• type (1:1 / group)
• createdAt
• lastMessageSeq
• status

👉 Used for chat list + quick access to latest message

ConversationMember

• conversationId
• userId
• role
• joinedAt
• status

👉 Defines group membership

UserConversationState (important)

• userId
• conversationId
• lastReadSeq
• lastDeliveredSeq

👉 Tracks read/delivery state efficiently (instead of per-message status)

Detailed Component Design

We will now deep dive into:

Ensure correct message ordering

• Mesages can arrive out of order due to network latency, retries or any other reason, we need to ensure correct ordering of messages in a conversation.

Approach:

1. Kafka Ordering
   • key = conversationId
   • ensures all messages in a conversation go to same partition
   • guarantees ordered processing
2. DB ordering:
   • Each message is assinged a monotonically increasing sequence number

Chat service assings to a message:

• • • ConverstationID
    • SequenceNumber

Implementation Option:

• • • Use Redis INCR for sequence number -> fast and simple
    • Eg: conversationId -> seqNumber
    • So when a message is to be created. use redis to fetch the next seqNumber
    • Then save the message in Db
    • Client that sorts message by converstationId + seqNumber

Ordering is guaranteed by Kafka partitioning + monotonic sequenceNumber in DB

Group Message flow

Steps:

1. user sends messages -> websocket gateway -> chat service
2. Chat service assigns seqnumber
3. Chat service stores the message to DB
4. Emit MessageCreatedEvent to kafka
5. Notification service -> sends push noticiation if user offline
6. Delivery service ->
   1. Consumes message created event
   2. Resolves group members
   3. checks presence
   4. pushes via socket connection

Group Scaling

• Group chats introduces a fan out problem, one message to be delivered to thousands of users
• This leads to
  • High latency
  • Server overload
  • High CPU usage

Solution:

• Firstly we limit max group size to be 1000,this way system dont have to deal with higher numbers such as 10k,
• Secondly we use fanout strategies. We use hybrid approach
  • Smaller groups - fanout on writes
    • When a message is created, The message is written per user
    • Each user gets their own copy of message
    • Fast reads, expensive writes
  • Larger grops - fanout on read
    • When a message is created, the messasge is written once to the group
    • Users fetch messages when opening chat
    • Cheap writes, expensive reads

Optimization oppurtunities:

• partition groups by Conv
• Avoid sync fanout in chat service
• Batch delivery via kafka consumers

WebSocket Scaling

Problem: Millions of active users = millions of connections, so scaling is ciritical

• Ensuring user stay on the same gateway during their active connection
  • Use LB with consitent hashing, so user goes to the same gateway all the time
• Ensure messages routes to correct server -> Solved via Presence Service + Redis

Actual Scaling:

• Horizopntal scaling: Add more gateways
• Add load balancers to distribute connections

Failure handling:

• If gateway fails:
  • Connecitons drops
  • user reconnects
  • Presence service updates new gateway connection