Requirements

Functional Requirements

1. User should be able to upload a file.
2. User should be able to share a file with other users.
3. User should be able to download the file.
4. User should be able to delete the file.
5. The files uploaded by the user must be synced across all devices where the user is connected via to the system.

Non functional Requirements

1. Consistency - The system should be fairly consistent. Two users trying to edit the same file, must see the recent changes to the file.
2. Availability - The system must be highly available, in case of failures etc.
3. Low Latency - The system should make all the operations - upload, delete, download, sync as fast as possible.
4. The system should support uploading of larger files upto 50 GB.

Define Core Objects

Core Entities

1. A File - File is the basic entity, that will be uploaded by user and shared / downloaded etc.
2. File Metadata - Information about the file like size of the file, type of file, user who uploaded, date of creation etc.
3. User - User who uploaded the file

Core Objects that will be needed to design the system

1. The system will have some kind of a UI where the user will be able to upload the file at the client side.
2. Uploader - at the Client side to chunk the file and upload the file
3. Downloader - at the client side to download the file.
4. APIs / API Gateway - Where the user will interact and make all requests related to the file operations.
5. Load Balancers and Rate Limiters to route the requests from users to application servers.
6. Application servers to handle request processing. This will include something like a file service which is responsible for handling the requests for all the file operations.
7. Database - to store the file information and user related information.
8. A blob Storage to store the actual files. This Blob storage can be something like S3.

Analyze Relationships

1) User should be able to upload a file to the system

1. The user uploads a file.
2. The uploader sends the request to the application server based file service.
3. The request goes through a rate limiter and load balancer to route the request to the best server to handle the request.
4. The file service sends a request to the s3, for a presigned URL which will be directly used by the uplaoder to upload the file directly to S3.
5. Once the file is uploaded on S3, S3 will send an event notification to the backend via the SNS about the completion of the file upload and the backend will update the file metadata table marking the status of the file as complete and uploaded.

2) User should be able to download a file from the System

1. The Downloader will use the presigned URL to request the file to download from the CDN.
2. Using a CDN improves the latency, and download times. Since it will serve the user with a copy of the file from the server that's located closest to the user.

3) User should be able to share a file with another user.

1. We need to be able to store the users with access to the file Ids, some kind of a user share list in the database.
2. The best way is to create a table with userId-fileId, which will store all the users and the files they are allowed to access.
3. The above way will help for fast lookups and detemrine, which users are allowed to edit/download files.

4) Users should be able to sync the file changes across all devices they are connected on

1. There will be two types of sync - Local to Remote Sync and Remote to Local Sync
2. For Local to Remote sync, a client side sync agent can be used to monitor for changes to the file and can queue them for uplaod locally. Then it can use the Upload API to upload the file.
3. For Remote to Local Sync, a hybrid approach of Polling + Web Sockets can be used. For Files that have been recently edited, (we can agree on waht recent means here - could be a week ir few hours), web sockets can be used for a real time sync. For files that are less frequently edited, we can use long polling to update the files only when there are changes in the server version.

Establish Hierarchy

Design inheritance trees where applicable to promote code reuse and polymorphism. This step involves identifying common attributes and behaviors that can be abstracted into parent classes...

Design Patterns

Consider using design patterns (e.g., Factory, Singleton, Observer, Strategy) that fit the problem...

Define Class Members (write code)

Attributes: For each class, define the attributes (data) it will hold...

Methods: Define the methods (functions) that operate on the attributes. Ensure they align with the object's responsibilities and adhere to the principle of encapsulation.

Adhere to SOLID Guidelines

Check and explain whether your design adheres to solid principles (Ask interviewer what SOLID principle is if you can not recall it.)...

Consider Scalability and Flexibility

Handling Large File Uploads upto 50 GB

We can break the file into chunks via a process called chunking. A large file can be broken down into chunks of like 5-10 MB and this will be then uploaded via the uploader. The chunking will be done at the client side. The chunks information i.e, all the chunks related to a particular file can be stored in the file metadata table. Based on the chunks that have been uploaded, the user can also be shown the progress as a percentage on the FrontEnd.

The File will be marked as uploaded once all the chunks for the resulting file have been uploaded by S3.

In order to uniquely identify chunks associated with File, for example many files will be uploaded concurrently by different users, and we need to be able to track the chunks belonging to a particular file. We can Use the concept of Fingerprinting - Fingerprinting is the process of computing a hash function based on the content of the file using a secure hash algorithm like SHA-256. We can hash the chunks and the file and store that information (hashed FileId and related chunks in the database) . This helps us to uniquely identify files and associated chunks.

Create/Explain your diagram(s)

Try creating a class, flow, state and/or sequence diagram using the diagramming tool. Mermaid flow diagrams can be used to represent system use cases. You can ask the interviewer bot to create a starter diagram if unfamiliar with the tool. Briefly explain your diagrams if necessary...

Future improvements

Critically examine your design for any flaws or areas for future improvement...