Requirements
Functional Requirements:
- Allow users to upload files to the system.
- Enable users to download uploaded files.
- Ensure synchronization of files between local and server storage.
- Batch upload & download
Non-Functional Requirements:
- app availability 99.99%, 52min downtime in a year
- file/data eventual consistency
- scalability, when a server fails, the failover
- security, only user with permission can upload/view. encryption for file object
world 8B, 10% users from pop = 0.8B
system DAU will be 10% of users = 80M
R:W = 10:1
W DAU
API Design
1x1
POST /v1/file {file info}-> return {status, error code, location url}
batch
POST /v1/files {files info}-> return {status, error code, location url}
download a file
GET /v1/file/id
GET /v1/files/id={id1, id2,..}
High-Level Design
fileInfo: fileId, name, auther, updated, created
chunkInfo: fileId, chunkId, chunkUrl
Detailed Component Design
Failover:
Active-Active:
Write to primary db, read from local.
mutli-region :
Quotrum: ex. 5 servers, only 3 server got replication, then the upload operation is called success.
During failover, use the latest replication lag server as a primary.
local:
Wait for replication, only if 1 server got success and write to replicate delay log, then it's called success. It speed up read access
Hot file:
Use fileInfo cache by redis