Design a large-scale File Storage Platform

Question

Design a fault-tolerant file storage system that handles millions of requests. Discuss trade-offs in consistency, availability, and performance.

Codemia · Accepted Answer

- **Functional Requirements:**  
  1. Users can upload, download, and delete files.  
  2. Support file sharing between users with permission settings.  
  3. Provide metadata for files (e.g., upload time, file size, file type).  
  4. Implement version control for files.  
  5. Allow for searching and filtering files by metadata.

- **Non-Functional Requirements:**  
  1. **Scalability:** Must handle millions of file requests per second.  
  2. **Fault Tolerance:** Ensure data is not lost in case of server failures.  
  3. **Availability:** System should be available 99.99% of the time.  
  4. **Performance:** Latency for file operations should be under 100ms.  
  5. **Security:** Implement encryption for files both in transit and at rest.

Assuming Snapchat has around 400 million daily active users, and each user uploads an average of 3 files per day, we can estimate:  
- **Total Daily Uploads:** 400M users * 3 files/user = 1.2 billion uploads/day  
- **Total Requests Per Second (RPS):** 1.2 billion uploads / (24 * 60 * 60) = ~13,888 uploads/second.  
- **Considering downloads and deletions, let’s assume a 3:1 ratio of reads to writes, leading to a total of 55,555 RPS.**  
- **Storage Needs:** If the average file size is 5MB, total data stored would be approximately 1.2 billion files * 5MB = ~6TB per day, leading to around 2.2PB of data in a year.

The architecture consists of the following components:  
1. **Client Layer:** Mobile and web clients for uploads and downloads.  
2. **API Gateway:** Handles incoming requests and routes them to appropriate services.  
3. **File Storage Service:**  
   - Uses **Amazon S3** for durable file storage.  
   - **Redis** for caching frequently accessed metadata.  
4. **Database Layer:**  
   - **Cassandra** for storing file metadata due to its scalability and write performance.  
   - Table design includes:  
     - `Files`: (file_id, user_id, upload_time, file_size, file_type, version)  
     - `FilePermissions`: (file_id, user_id, permission)  
5. **Monitoring & Logging:** Use **Prometheus** for monitoring and **ELK Stack** for logging to track system health and performance.

The schema design for our file storage system would include the following tables:  
- **Files Table:**  
  - `file_id` (UUID, Primary Key)  
  - `user_id` (UUID, Foreign Key)  
  - `upload_time` (Timestamp)  
  - `file_size` (Integer)  
  - `file_type` (String)  
  - `version` (Integer)  
- **FilePermissions Table:**  
  - `file_id` (UUID, Foreign Key)  
  - `user_id` (UUID, Foreign Key)  
  - `permission` (String)  
This model supports efficient querying of files by user and enables version control and permission management.

1. **Consistency vs. Availability:**  
   - Chose eventual consistency for file metadata to improve availability. Users may see stale data for a few seconds but it allows for higher availability during high traffic.  
2. **Storage vs. Performance:**  
   - Used S3 for storage due to its durability and scalability, but it may introduce latency in accessing files. Caching with Redis mitigates this for frequently accessed files.  
3. **Complexity vs. Scalability:**  
   - The architecture is designed to be horizontally scalable by adding more instances of services (e.g., File Storage Service) as load increases. However, this adds complexity in managing distributed data and consistency.

Design a large-scale File Storage Platform

Snapchat

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Practice a Similar Problem on Codemia

Sample Answer

Requirements

Capacity Estimation

Submit Your Answer

Snapchat Software Engineer Interview Guide

Related Questions

Design a high-throughput Inventory Management System

Design a low-latency Rate Limiting System

Design a fault-tolerant Payment System

Design a fault-tolerant Messaging System

Design Walmart Product Search