System requirements

Functional:

1. Client should be able to collaborate editing same file.
2. Client should be able to upload files.
3. Client should be able to download files.
4. Client should be able to create folder to group and store files.

Non-Functional:

1. Low latency.
2. High availability.
3. Scalability.
4. Eventually consistency.
5. Security.

Capacity estimation

Assume 1B daily active users

Read QPS: 1B / 100k = 10k

Peak read QPS: 2 * 10k = 20k

Assume 10% user are editing files

Write QPS: 1B * 10% / 100k = 1k

Peak write QPS: 2 * 1k = 2k

Storage usage:

Assume each user will write average 100kb daily. So the daily storage usage is 1B * 100kb = 100TB. Yearly storage usage is 400 * 100TB = 40PB. Each data will need 3 replicas. The the total yearly storage needed is 3 * 40PB = 120 PB.

API design

GET getUserFiles(userId, folderId)

GET getFileMetadata(userId, fileId)

POST uploadFile(fileId, fileChunks)

GET downloadFile(fileId, fileChunkId)

WebSocket API for cooperate editting:

From client to server:

editFile(fileId, editDetailInfo)

From server to client:

editFile(fileId, editDetailInfo)

Database design

See Diagram

High-level design

See Diagram

Request flows

See Diagram

Detailed component design

See Diagram

Trade offs/Tech choices

See Diagram

Failure scenarios/bottlenecks

Our non-functional requirements for the proposed system design are scalability, fault tolerance, availability, and low latency. Let’s discuss how the proposed system fulfills these requirements:

1. Scalability: The proposed system is scalable to handle an ever-increasing number of users. The required resources, including load balancers, web servers, and other relevant servers, are added/removed on demand.
2. Fault tolerance: The replication of data of both File Blob storage database and File Metadata NoSQL DB makes the system fault-tolerant. Moreover, the redundant resources are always there to handle the failure of a server or its component. Monitoring service is used to enhance reliability by continuously observing system health, detecting issues early, providing insights for optimization, and assisting in timely incident response.
3. Availability: The system is highly available by providing redundant servers and replicating data on them. When a user gets disconnected due to some fault in the server, the session is re-created via a load balancer with a different server. Moreover, the data (file metadata, file blob storage) is stored on different and redundant database clusters, which provides high availability and durability.
4. Low latency: We can minimize the system’s latency at various levels by:

• Geographically distributed servers and the cache associated with them. This way, we bring the service close to users.
• Using CDNs for frequently accessed file blob content.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?