Loading...
Estimate the scale of the system. Consider daily active users, read/write ratio, storage requirements, bandwidth, and any relevant QPS calculations...
Define the APIs expected from the system. This is your chance to analyze and define the read and write paths so that you can come up with the high-level design...
The API is designed around chunking from the ground up. Files never move as monolithic blobs. Every upload and download operates on 4MB chunks, enabling parallel transfers, deduplication, and resumability.
Phase 1: Initiate upload and dedup check
POST /v1/files/upload/init
{
"name": "presentation.pptx",
"parent_folder_id": "folder_abc",
"total_size": 52428800,
"chunk_hashes": [
"sha256:a3f2b8c...",
"sha256:7d1e9f4...",
"sha256:b5c8a2e...",
...
]
}
Returns:
json
{
"upload_session_id": "sess_xyz",
"missing_chunks": [0, 2, 7, 11],
"presigned_urls": {
"0": "https://s3.../sha256:a3f2b8c?X-Amz-Signature=...",
"2": "https://s3.../sha256:b5c8a2e?X-Amz-Signature=...",
...
}
}
Phase 2: Upload missing chunks
PUT {presigned_url}
Content-Type: application/octet-stream
Body: <raw chunk bytes>
Chunks upload directly to S3 via pre-signed URLs, bypassing the application servers entirely. The client uploads multiple chunks in parallel (typically 4-8 concurrent uploads). Each completed upload is confirmed by S3 with an ETag.
Phase 3: Commit
POST /v1/files/upload/commit
{
"upload_session_id": "sess_xyz",
"chunk_etags": {"0": "etag_abc", "2": "etag_def", ...}
}
The server verifies all chunks are present, creates a new file version record in PostgreSQL linking to the chunk hashes, increments the version counter, and publishes a sync event to Kafka. The file is now visible to other devices.
GET /v1/sync/changes?cursor={cursor}
Returns a batch of file changes (creates, edits, deletes, moves) since the cursor position. If no changes are pending, the server holds the connection open for up to 90 seconds (long polling) before returning an empty response. The cursor is an opaque token encoding the last-seen change sequence number.
Why long polling instead of WebSockets? Long polling works through corporate proxies and firewalls that often block WebSocket upgrades. For a file sync use case where changes arrive every few minutes (not every second), the overhead of reconnecting every 90 seconds is negligible.
GET /v1/files/{file_id}/download?version={version}
Returns chunk hashes and pre-signed S3 download URLs. The client downloads chunks in parallel, verifies SHA-256 hashes locally, and assembles the file. For files already partially cached (delta sync scenario), the client only downloads chunks it doesn't already have.
POST /v1/files/share, Share a file or folder with specified users and permission level (view, edit, comment).
GET /v1/files/{file_id}/versions, List version history with timestamps, editor, and size delta.
POST /v1/files/move, Move or rename a file/folder. Updates parent_folder_id and triggers sync notification.
Describe the overall system architecture. Identify the main components needed to solve the problem end-to-end. Use the diagramming tool to create a block diagram.
Client apps (desktop, mobile, web): The desktop client is the most sophisticated: it watches the local Dropbox folder for file changes, computes chunk hashes using SHA-256, maintains a local database of known chunk hashes for delta detection, and manages the upload/download queue. Mobile clients sync on-demand (pull individual files) rather than syncing the entire folder. The web client provides browser-based access with drag-and-drop upload.
API Gateway: Single entry point handling authentication (OAuth 2.0 tokens), rate limiting (token bucket per user), TLS termination, and request routing to backend services. The gateway also handles the long-poll connections for sync notifications, keeping connections open for up to 90 seconds.
File Service: Manages file metadata operations: create, rename, move, delete, share, and version listing. Reads and writes to PostgreSQL. Publishes file change events to Kafka for downstream consumers (Sync Service, CDN invalidation). Stateless. Any instance can handle any request.
Chunk Service: Handles the chunk-level operations: dedup checking (hash lookup against the chunks table), pre-signed URL generation for S3 uploads, upload session tracking, and commit verification. This service never handles chunk data directly. It only generates URLs that let clients talk to S3.
Key Insight
The Chunk Service never touches file bytes. It generates pre-signed S3 URLs and tracks upload sessions. This separation means application servers handle only lightweight metadata operations while S3's infrastructure handles the bandwidth-intensive data transfer. At 20TB/day of incoming data, this avoids routing terabytes through your own servers.
Sync Service: Maintains long-poll connections with clients and pushes change notifications when files are created, modified, or deleted. Consumes events from Kafka and fans them out to connected clients. Each client provides a cursor (sequence number) and the Sync Service returns all changes since that cursor. Uses consistent hashing to assign user-to-server affinity so a user's notification state lives on one Sync Service instance.
PostgreSQL: Stores all structured metadata: users, files, file versions, chunks (hash + ref_count), version_chunks, and permissions. Deployed with synchronous replication to a standby for zero data loss on primary failure. Read replicas handle the read-heavy metadata queries.
S3 (object storage): Stores chunk data with the SHA-256 hash as the object key. Content-addressable: the same bytes always map to the same key. Cross-region replication for durability. Lifecycle policies archive old versions to S3 Glacier after 90 days.
Kafka: Event bus connecting services. When a file is uploaded, the File Service publishes a change event. The Sync Service consumes it to notify clients. Other consumers handle audit logging, analytics, and CDN cache warming. Kafka decouples producers from consumers and absorbs traffic spikes during peak hours.
Redis: Distributed locking and lease management for Sync Service instances. Each instance holds a time-limited lease on its assigned user partitions, preventing split-brain scenarios during failover. Leases are stored with a TTL so that if an instance crashes, its lease expires automatically and another instance takes over.
CDN: Caches popular shared files at edge locations. When a file is shared with thousands of users (e.g., a company-wide document), the CDN serves subsequent downloads without hitting S3. Most personal files are not CDN-cached because they are accessed from a small number of devices.
Define the data model. Identify the main entities, their attributes, and relationships. Consider the choice of database type (SQL vs NoSQL) and justify your decision based on access patterns...
The data model separates into two distinct stores: PostgreSQL for structured metadata (files, versions, permissions, chunk references) and S3 for the actual chunk data. This split reflects their fundamentally different characteristics, metadata needs ACID transactions and complex queries while chunk data needs cheap, durable blob storage.
sql
-- Users
CREATE TABLE users (
user_id UUID PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
display_name VARCHAR(100),
storage_quota BIGINT DEFAULT 2147483648, -- 2GB default
storage_used BIGINT DEFAULT 0,
plan_type VARCHAR(20) DEFAULT 'free',
created_at TIMESTAMPTZ NOT NULL
);
-- File tree (files and folders)
CREATE TABLE files (
file_id UUID PRIMARY KEY,
owner_id UUID REFERENCES users(user_id),
parent_folder_id UUID REFERENCES files(file_id),
name VARCHAR(255) NOT NULL,
is_folder BOOLEAN DEFAULT FALSE,
current_version INTEGER DEFAULT 1,
deleted_at TIMESTAMPTZ,
UNIQUE(parent_folder_id, name)
);
-- Version history
CREATE TABLE file_versions (
version_id UUID PRIMARY KEY,
file_id UUID REFERENCES files(file_id),
version_num INTEGER NOT NULL,
size_bytes BIGINT NOT NULL,
editor_id UUID REFERENCES users(user_id),
created_at TIMESTAMPTZ NOT NULL,
UNIQUE(file_id, version_num)
);
-- Content-addressable chunk registry
CREATE TABLE chunks (
chunk_hash VARCHAR(64) PRIMARY KEY, -- SHA-256 hex
size_bytes INTEGER NOT NULL,
ref_count INTEGER DEFAULT 1,
created_at TIMESTAMPTZ NOT NULL
);
-- Links versions to their chunks (ordered)
CREATE TABLE version_chunks (
version_id UUID REFERENCES file_versions(version_id),
chunk_index INTEGER NOT NULL,
chunk_hash VARCHAR(64) REFERENCES chunks(chunk_hash),
PRIMARY KEY(version_id, chunk_index)
);
-- Sharing permissions
CREATE TABLE permissions (
permission_id UUID PRIMARY KEY,
file_id UUID REFERENCES files(file_id),
user_id UUID REFERENCES users(user_id),
access_level VARCHAR(20) NOT NULL, -- view, edit, comment
granted_at TIMESTAMPTZ NOT NULL,
UNIQUE(file_id, user_id)
);
-- Upload session tracking (for resumable uploads)
CREATE TABLE upload_sessions (
session_id UUID PRIMARY KEY,
file_id UUID REFERENCES files(file_id),
uploader_id UUID REFERENCES users(user_id),
total_chunks INTEGER NOT NULL,
status VARCHAR(20) DEFAULT 'in_progress',
created_at TIMESTAMPTZ NOT NULL,
expires_at TIMESTAMPTZ NOT NULL -- 24-hour TTL
);
-- Tracks which chunks are confirmed per session
CREATE TABLE session_chunks (
session_id UUID REFERENCES upload_sessions(session_id),
chunk_index INTEGER NOT NULL,
chunk_hash VARCHAR(64) NOT NULL,
etag VARCHAR(64) NOT NULL,
uploaded_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY(session_id, chunk_index)
);
Chunks are stored in S3 with the SHA-256 hash as the object key: s3://dropbox-chunks/{chunk_hash}. This is content-addressable storage. The key is derived from the content itself. Two identical chunks uploaded by different users produce the same key and are stored exactly once.
The ref_count on the chunks table is critical for garbage collection. When a file version is deleted (e.g., expired from version history), the system decrements the ref_count of each referenced chunk. When ref_count reaches zero, no file version references that chunk, and it can be safely deleted from S3. This prevents both premature deletion (a chunk still used by another file) and orphaned storage (chunks that no one references).
The files table uses a parent_folder_id self-reference to model the directory tree. The UNIQUE constraint on (parent_folder_id, name) prevents duplicate filenames within a folder, enforcing file system semantics at the database level.
Deep dive into 2-3 key components. Explain how they work, how they scale, discuss tradeoffs, capacity, and any relevant algorithms or data structures.
Every chunk is identified by its SHA-256 hash. This hash serves triple duty:
s3://dropbox-chunks/{sha256_hash}. Two users uploading identical files produce identical hashes and map to the same S3 object.Key Insight
Dedup happens at two levels. Cross-user dedup: 1,000 employees receive the same email attachment and save it to Dropbox (stored once. Cross-version dedup: a user edits a 100MB file, changing one chunk) the 24 unchanged chunks are not duplicated in the new version. Both levels use the same mechanism: SHA-256 hash lookup in the chunks table.
SHA-256 collision risk: The probability of two different chunks producing the same SHA-256 hash is approximately 1 in 2^256 (roughly 10^77). For context, if you hashed every atom in the observable universe, the chance of a collision would still be negligible. No file storage system has ever observed a SHA-256 collision. This is not a theoretical concern in practice.
Conflict resolution: version counter detects concurrent edits, creating a conflicted copy for manual resolution
When two devices edit the same file concurrently:
report (Device B's conflicted copy 2026-03-03).docx. The original file syncs to version 6 (Device A's changes).Why not auto-merge? Dropbox handles arbitrary binary files (PDFs, images, executables), not just text. Auto-merging two versions of a JPEG or a compiled binary is meaningless. Even for text files, automatic merging requires understanding the file format (XML, JSON, plain text each have different merge semantics). The conflicted copy approach is format-agnostic and always safe. No data is ever lost.
Common Pitfall
Conflict resolution is the GATE topic for this problem. Interviewers expect you to explain why Dropbox uses conflicted copies instead of auto-merge, how the version counter detects conflicts, and what happens to the losing device's changes. Saying 'last write wins' is insufficient, you must explain that the conflicted copy preserves both versions so no data is lost.
Level Expectations
Mid-level: explain fixed-size chunking and basic dedup via SHA-256 hash lookup.
Senior: compare fixed-size vs content-defined chunking with specific examples (what happens when a byte is inserted), explain the version counter conflict detection mechanism, and discuss ref_count garbage collection.
Staff: design the Rabin fingerprint boundary detection algorithm, analyze SHA-256 collision probability in context, and propose a conflict resolution UX that minimizes user friction (automatic merge for known text formats, conflicted copy for binary).