Loading...
message is a chat system while the user:
1, 1:1 chat 1:n group chat, user could send message to 1 user or several users
2,create/delete the group chat, manage the other users on the group
3,send/view messages to the group
4,message wiil store for 1 year
5,support text/media, including the video and the image information
1, support the web/mobile side
2, high availability
3, medium data consistency, eventually consistent
4,network failure tolenrance
5, scalable
1, daily ative user: 500m
2, each user 10mins on the chat, 2kB text /per, 1MB media/per
500m*2KB= 1TB, the data center size
500m*1MB=500TB,
assuming the compression ratio, 5:1, 100 TB /day-> 36 PB year
replication, 144PB a year
POST create_group(sender_userid): returns the group id
GET join_group(sender_id, group_id): establish the websocket connection
POST upload_media(sender_id, group_id, metadata):return the pre_assigned url
message use JSON format:
{ "message_id": 1
"message": ""
"media_url": "url"
}
So when we're talking about a database, the first thing is to choose the database. There are several options.
One is the relational database, and the second will be the non-SQL databases. For our scenario, because we don't want our, we don't need relational properties like the ACID property. So we don't need relational databases at this time.
And for the non-SQL databases, it has several advantages, like the non-SQL DB such as MongoDB. It's very fixed for a document to store the text data. And also, it's scalable. If the size of users increases, it's quite suitable for such a situation. And we also have other choices for the non-SQL DB databases, like the document database, like the wide column database, such as Cassandra. The Cassandra would be even more scalable than MongoDB. But its size may not fit this range.
MongoDB's schema is very flexible. So if you want to add more columns to the database, it will be good in the future for our product enhancement.
The wide column DB is horizontally scalable, its rigid schema would lack of indexing support, and it is not suitable for the data model.
So at this time, we would like to choose MongoDB.
Data model:
users (
user_id VARCHAR(50) PRIMARY KEY,
username VARCHAR(100) NOT NULL,
email VARCHAR(100) NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
messages (
message_id VARCHAR(50) PRIMARY KEY,
sender_id VARCHAR(50) NOT NULL,
receiver_id VARCHAR(50) NOT NULL,
timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
type ENUM('text', 'image', 'video') NOT NULL,
content TEXT,
media_link VARCHAR(50) NOT NULL,
FOREIGN KEY (sender_id) REFERENCES users(user_id),
FOREIGN KEY (receiver_id) REFERENCES users(user_id)
);
1, timestamp we would use the the server side's clock, easy to sync
2, the data should be indexed by the message id, sender and the timestamp
3, user's geo location, since the messages are mostly locally. we would store the message to the local data center.
To improve its fault tolerance, we would make replicas, the replicas would be in other regions.
API Gateway:
Load Balancer:
Microservices:
MessageQueue:
Media Worker: Processes media files, generates multiple versions, and stores URLs in MongoDB.
Client Side:
client connection
The user opens the chat application and establishes a connection to the server, usually via WebSocket for real-time communication.
The client sends a connection request to the WebSocket server.
Authentication
The WebSocket server receives the connection request and authenticates the user through an API gateway or authentication service (such as through a JWT token).
After successful authentication, the server confirms the connection and starts allowing messages to be sent and received.
Send a message
Users send messages (text, images, videos, etc.) on the client side.
The client encapsulates the message into a specific format (such as JSON) and sends it to the server through the WebSocket connection.
Message processing
The WebSocket server receives the message and performs preliminary processing, such as verifying the message format and content.
The server stores message metadata (such as sender ID, receiver ID, timestamp, message type, etc.) to the database.
For image and video messages, the server uploads these files to the CDN and obtains the URL of the file.
message distribution
The server distributes messages to recipients over WebSocket connections.
If the recipient is online, the message is immediately pushed to the recipient's client via WebSocket.
If the recipient is offline, the server stores the message in the offline message table in the database and will push it when the recipient comes online.
Message confirmation
After receiving the message, the recipient's client sends an acknowledgment (ACK) to the server.
The server updates the message status (such as from "Sent" to "Received" or "Read").
Message synchronization
If the recipient is logged in on multiple devices, the server syncs the message to all devices.
Use a device management service to ensure consistent messaging status for all devices.
Let's discuss how we can make sure this architecture has good performance and scalability.
I expect creating groups and starting chat can perform well (100ms range). Data models are simple, I don't see complex joins, and all the data can be cached by Redis. There should be strong locality of access.
One key for good user response time would be WebSocket. Fortunately, it should be horizontally scalable. Each client establishes a WebSocket connection with one WebSocket server. Load Balancer makes sure the ensuing packets should go to this server. As the number of clients and requests increase, we can add more WebSocket servers.
WebSocket Service uses Pub/Sub queue for efficient communication. For example, let's say Server A who has WebSocket connection open with User A. User A sends a message. Server A would publish this message in Pub/Sub server. Service B and Server C are listening to a Pub/Sub topic which represents a chat group. Upon receiving User A's message, Servers B and C send this message to User B and C, respectably, via WebSocket.
Another key for good response time is media. I did not have time to draw it in this architecture, but we should implement adaptive streaming protocol (e.g. DASH) between a service (Streaming Service) and the client. To support this, Media Worker should generate multiple versions (sizes and formats) of an uploaded video, and store them in Blob Store and in Cache.
Real-time Performance:
Technology: WebSocket or Long Polling for Real-time Messaging
Implementation: Each user maintains a persistent connection with a specific server cluster close to their geographical location.
Pros: Ensures low latency by minimizing the physical distance between the user and the server.
Cons: Managing these connections and ensuring real-time performance becomes challenging as the number of users increases globally.
Scalability:
Technology: Distributed Architecture (e.g., Microservices, Distributed Databases)
Implementation: Messages are routed through a distributed system that can dynamically scale. Use technologies like Apache Kafka for messaging, Redis for caching, and a distributed database like Cassandra.
Pros: System can handle millions of users by scaling horizontally. Each microservice can be scaled independently based on load.
Cons: Increases complexity significantly. Ensuring real-time performance while maintaining consistency and availability across distributed nodes can be complex and requires robust infrastructure.
1. Network Failures
Failures:
Disconnected Clients: Users lose their network connection, causing disconnection from the chat server.
Network Latency: High latency can lead to delayed message delivery and poor user experience.
Mitigation Strategies:
Reconnection Logic: Implement automatic reconnection logic on the client side. When a connection is lost, the client should periodically attempt to reconnect.
Heartbeat Messages: Use heartbeat or ping-pong messages to detect inactive connections and clean them up.
Local Caching: Cache unsent messages locally on the client and retry sending them once the connection is restored.
Quality of Service (QoS): Implement QoS policies to prioritize critical messages and handle retries.
2. Server Failures
Failures:
Server Crashes: Chat server crashes due to software bugs, hardware failures, or resource exhaustion.
Overloaded Servers: High traffic causes the server to become overloaded and unresponsive.
Mitigation Strategies:
Load Balancing: Use load balancers to distribute incoming connections across multiple servers.
Auto-Scaling: Implement auto-scaling policies to dynamically add or remove server instances based on traffic.
Graceful Degradation: Design the system to degrade gracefully under high load, such as limiting the number of concurrent connections or temporarily disabling non-critical features.
Health Checks and Monitoring: Continuously monitor server health and set up automated alerts and recovery mechanisms (e.g., restarting failed instances).
3. Data Loss
Failures:
Message Loss: Messages are lost due to network issues, server crashes, or bugs.
Database Failures: Data corruption or loss in the database storing chat messages.
Mitigation Strategies:
Message Acknowledgements: Implement message acknowledgements to ensure messages are successfully received and processed by the server.
Durable Storage: Use a highly available and durable storage system (e.g., distributed databases like Cassandra or DynamoDB) to store chat messages.
Replication and Backups: Implement data replication and regular backups to prevent data loss and facilitate recovery.
4. Consistency Issues
Failures:
Message Ordering: Messages appear out of order due to network latency or server processing delays.
Duplicate Messages: Messages are delivered more than once due to retries or network issues.
Mitigation Strategies:
Unique Message IDs: Assign unique IDs to each message to detect and discard duplicates.
Sequence Numbers: Use sequence numbers or timestamps to maintain message ordering. The client can reorder messages based on these numbers.
Eventual Consistency: Design the system for eventual consistency, ensuring that all replicas converge to the same state over time.
What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?