System requirements

Functional:

• File operation(Read/Write/Append/etc.)
• Directory operation.
• Security.

Non-Functional:

• Availability
• Consistency
• Fault tolerance
• Reliability

Capacity estimation

Estimate the scale of the system you are going to design...

API design

# REST Examples

GET /file/path/to/file.txt

PUT /file/path/to/file.txt

POST /file/path/to/file.txt?append=true

DELETE /file/path/to/file.txt

Database design

File Representation

• Files are split into fixed-size chunks (e.g., 64MB - 256MB).
• Each chunk has a unique ID (UUID).
• Files and directories stored in a hierarchical namespace maintained by the Metadata Server.

Chunk Storage

• Each chunk is replicated across R storage servers (e.g., default R=3).
• Stored using local file systems (ext4, xfs).

nodes Table – Files and Directories

TABLE nodes (

id,

name ,

parent_id,

is_directory,

size,

created_at,

modified_at,

permissions,

owner_id UUID,

);

chunks Table – Logical Data Blocks

TABLE chunks (

id,

node_id,

chunk_index,

checksum,

size,

);

locks Table – Distributed Locking

TABLE locks (

node_id ,

client_id,

lock_type TEXT CHECK (lock_type IN ('read', 'write')),

acquired_at,

expires_at ,

PRIMARY KEY (node_id)

);

High-level design

• API gateway: Interface through which users/applications interact with the system.
• Metadata Server: Maintains metadata like file hierarchy, chunk mappings, and permissions.
• Chunk Servers: Store actual file data in fixed-size chunks.
• Distributed Locking: Ensures consistency and coordinates concurrent access.
• Replication: Each chunk is stored on multiple chunk servers for fault tolerance.
• Heartbeats: Chunk servers send periodic status updates to the Metadata Server.

Request flows

Explain how the request flows from end to end in your high level design. Also you could draw a sequence diagram using the diagramming tool to enhance your explanation...

Detailed component design

Dig deeper into 2-3 components and explain in detail how they work. For example, how well does each component scale? Any relevant algorithm or data structure you like to use for a component? Also you could draw a diagram using the diagramming tool to enhance your design...

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Try to discuss as many failure scenarios/bottlenecks as possible.

Future improvements

What are some future improvements you would make? How would you mitigate the failure scenario(s) you described above?