System requirements

Functional Requirements:

• User registration, login, authentication.
• One-to-one messaging (text, emoji, attachments).
• Group chats.
• Message read/delivery status (sent, delivered, seen).
• Push notifications (mobile/web).
• Message search (by user or keyword).
• Media upload (images, videos).
• Presence indicator (online/offline).
• Message deletion/editing.
• Sync messages across devices (web + mobile).

Non-Functional Requirements:

• Low latency (<200 ms per message).
• High availability (99.99% uptime).
• Scalability (millions of users, billions of messages/day).
• Durability (no message loss).
• Security (end-to-end encryption, HTTPS, encryption at rest).
• Fault tolerance (data replication, failover).
• Consistency (eventual consistency across devices).

Capacity estimation

• DAU (Daily Active Users): ~100 million.
• Peak concurrent users: ~10 million.
• Average messages/user/day: ~50 → total 5 billion messages/day.
• Message size: ~1 KB.
• Storage/year: 5B × 1KB × 365 ≈ 1.8 PB/year.
• RPS (requests per second): ~60K–100K.

API design

POST /register → Register new user.

POST /login → Authenticate user.

GET /users/{id}/presence → Check online status.

POST /messages/send → Send message.

GET /messages/{conversation_id} → Fetch messages.

POST /groups/create → Create group.

POST /groups/{id}/add_member → Add user to group.

POST /messages/{id}/delete → Delete/edit message.

POST /messages/{id}/read → Mark message as read.

Database design

• User (id, name, email, password_hash, status, last_seen).
• Conversation (id, type [1:1, group], created_at).
• Participant (user_id, conversation_id, joined_at).
• Message (id, conversation_id, sender_id, content, type, created_at, status).
• Attachment (id, message_id, url, type, size).

Relationships:

• User ↔ Participant ↔ Conversation.
• Conversation ↔ Message.
• Message ↔ Attachment.

High-level design

API Gateway

Authentication Service

User Service

Conversation Service

Message Service

Notification Service

Presence Service

Media Service (file storage, CDN)

WebSocket Gateway / Message Broker

Database Cluster (SQL for metadata, NoSQL for message storage)

Cache Layer (Redis, Memcached)

Search Engine (Elasticsearch for message search)

Request flows

Send message:

1. Client → API Gateway → Authentication.
2. → Message Service → Store in DB.
3. → Push to Message Broker → Deliver to recipient(s).
4. → Update read/delivery status.
5. → Notify Notification Service.

Receive message:

1. WebSocket pushes message to client.
2. Client updates UI.
3. On read, client → API → Message Service → Update DB.

Detailed component design

Message Service:

• Write-optimized NoSQL DB (Cassandra, DynamoDB).
• Asynchronous writes to improve throughput.
• Write-through cache with Redis.
• Sharding by conversation_id.

WebSocket Gateway:

• Maintains persistent connections.
• Uses message broker (Kafka, RabbitMQ) for fan-out.
• Handles online/offline detection.

Notification Service:

• Sends push notifications (APNs, FCM).
• Deduplicates alerts.
• Integrates with presence service.

Trade offs/Tech choices

• WebSocket over polling → better latency, less resource waste.
• NoSQL for messages → scales horizontally, tolerates high writes.
• SQL for metadata → strong consistency for user, conversation data.
• Redis for presence → fast updates, ephemeral storage.
• Kafka → decouples producers/consumers, high throughput.

Failure scenarios/bottlenecks

WebSocket server crash → failover, reconnect from client.

Message broker lag → scale Kafka partitions.

Cache eviction → fallback to DB, monitor hit ratio.

Database hot partitions → shard carefully by conversation_id.

Future improvements

• End-to-end encryption.
• Video/audio calls integration.
• Message reactions.
• Better spam detection.
• Cross-region replication for latency reduction.