Design Dropbox

Functional

Dropbox is a cloud file storage and synchronization service that lets users save a file on one device and access it from any other device seamlessly. The deceptively simple promise ("save on laptop, appears on phone") hides serious engineering challenges around bandwidth efficiency, conflict resolution, and storage economics at scale.

• File upload and download: Users upload files of any type (documents, images, videos) up to 50GB each. Downloads retrieve the latest version. Both operations must be resumable. A 2GB upload interrupted at 80% should not restart from zero.
• Automatic sync across devices: When a user edits a file on their laptop, the change propagates to their phone, tablet, and web client without manual action. Sync must be bandwidth-efficient: editing one line of a 1GB file should not re-upload the entire gigabyte.
• Directory structure: Users organize files into nested folders. Moving, renaming, and deleting folders propagates to all contained files. Sharing permissions apply at the folder level and cascade to children.
• File sharing: Users share files or folders with others via link or by granting direct access. Permission levels include view-only, edit, and comment. Shared folders appear in each recipient's file tree.
• Offline editing: Users edit files without an internet connection. Changes queue locally and sync when connectivity returns. The system must handle conflicts when two users edit the same file offline.
• Version history: The system retains previous versions of every file (default 30 days, extended for paid plans). Users can view, compare, and restore any previous version.

We will explicitly defer real-time collaborative editing (Google Docs-style), full-text search across file contents, and file preview/thumbnail generation. These are important features but would expand the scope well beyond a 45-minute interview.

Non-Functional

• Durability (11 nines): Once a file is uploaded, the probability of data loss must be vanishingly small (99.999999999%). This drives the choice of S3-class storage with cross-region replication. Durability is the single most important property, users trust Dropbox with their only copy of critical files.
• Strong consistency for metadata: When a user deletes a file, every client must see the deletion immediately. Eventual consistency would cause a deleted file to reappear on another device, which is unacceptable for a file system. This pushes us toward a relational database for metadata.
• Bandwidth efficiency: This is the architectural driver. Users have limited upload bandwidth (typically 5-20 Mbps residential). A naive system that re-uploads entire files on every edit would be unusable for large files. Chunking, delta sync, and deduplication are not optimizations, they are core requirements.
• Availability (99.9%): File access should rarely be interrupted. Graceful degradation: if sync is down, users can still access locally cached files. Upload/download can tolerate brief outages because clients retry automatically.
• Scalability: Support hundreds of millions of users with billions of files totaling petabytes of storage. The system must scale horizontally at every layer.

Users and Files

• Total users: 500M registered, 200M monthly active, 100M daily active
• Average storage per user: 2GB across all files
• Total files: 100 billion files across all users
• Total stored data: 200M active users x 2GB average = 400PB of unique stored data. Most registered users are inactive or store very little. With versioning (30-day history adds roughly 20% overhead) and cross-region replication (3x), total physical storage footprint is much larger, but unique logical data is about 400PB. The dedup ratio on incoming data is approximately 50-60%, meaning we avoid storing roughly half of all incoming bytes.

Daily Traffic

• File writes: 20M file uploads or modifications per day (roughly 230 per second average, with 3-5x peaks during business hours)
• File reads: 200M file downloads/views per day (2,300/sec average). Read-heavy 10:1 ratio is typical for file storage, most files are written once and read many times.
• Raw data ingestion: 20M files/day x average 1MB = 20TB/day raw incoming data
• Net new storage after dedup: With a 60% dedup ratio on incoming data, roughly 8TB/day of net new unique data is written to S3. Over a year, that is about 2.9PB of new storage.

Key Cost Insight

At S3 standard pricing ($0.023/GB/month), 400PB costs roughly $9.2M/month in storage alone. The 60% dedup ratio on incoming data means roughly 12TB/day of avoided writes. Over one year, dedup avoids storing approximately 4.3PB of duplicate data. At $0.023/GB, that is roughly $100K/month in avoided storage costs, and the savings compound as the corpus grows. This justifies the entire complexity of the chunking and dedup system, content-addressable storage is not a clever optimization, it is an economic necessity. Without dedup, storage costs would grow 2.5x faster, making the business model unsustainable at scale.

Metadata

• File metadata size: Each file record (name, path, version, permissions, chunk references) averages 500 bytes
• Total metadata: 100B files x 500B = 50TB of metadata in PostgreSQL. This is large for a single cluster but manageable with table partitioning and read replicas.
• Daily metadata writes: 20M file operations x 500B = 10GB/day of metadata ingestion

The API is designed around chunking from the ground up. Files never move as monolithic blobs. Every upload and download operates on 4MB chunks, enabling parallel transfers, deduplication, and resumability.

Upload Flow (Three-Phase)

Phase 1: Initiate upload and dedup check

1POST /v1/files/upload/init
2{
3  "name": "presentation.pptx",
4  "parent_folder_id": "folder_abc",
5  "total_size": 52428800,
6  "chunk_hashes": [
7    "sha256:a3f2b8c...",
8    "sha256:7d1e9f4...",
9    "sha256:b5c8a2e...",
10    ...
11  ]
12}

Returns:

1{
2  "upload_session_id": "sess_xyz",
3  "missing_chunks": [0, 2, 7, 11],
4  "presigned_urls": {
5    "0": "https://s3.../sha256:a3f2b8c?X-Amz-Signature=...",
6    "2": "https://s3.../sha256:b5c8a2e?X-Amz-Signature=...",
7    ...
8  }
9}

The client sends all chunk hashes upfront. The server checks which hashes already exist in the chunk table (dedup check). It returns only the missing chunk indices and pre-signed S3 URLs for direct upload. If the file is an exact duplicate, missing_chunks is empty and no bytes are uploaded. The server just creates a new file record pointing to existing chunks.

Phase 2: Upload missing chunks

PUT {presigned_url}
Content-Type: application/octet-stream
Body: <raw chunk bytes>

Chunks upload directly to S3 via pre-signed URLs, bypassing the application servers entirely. The client uploads multiple chunks in parallel (typically 4-8 concurrent uploads). Each completed upload is confirmed by S3 with an ETag.

Phase 3: Commit

1POST /v1/files/upload/commit
2{
3  "upload_session_id": "sess_xyz",
4  "chunk_etags": {"0": "etag_abc", "2": "etag_def", ...}
5}

The server verifies all chunks are present, creates a new file version record in PostgreSQL linking to the chunk hashes, increments the version counter, and publishes a sync event to Kafka. The file is now visible to other devices.

Sync (Long Polling)

GET /v1/sync/changes?cursor={cursor}

Returns a batch of file changes (creates, edits, deletes, moves) since the cursor position. If no changes are pending, the server holds the connection open for up to 90 seconds (long polling) before returning an empty response. The cursor is an opaque token encoding the last-seen change sequence number.

Why long polling instead of WebSockets? Long polling works through corporate proxies and firewalls that often block WebSocket upgrades. For a file sync use case where changes arrive every few minutes (not every second), the overhead of reconnecting every 90 seconds is negligible.

Download

GET /v1/files/{file_id}/download?version={version}

Returns chunk hashes and pre-signed S3 download URLs. The client downloads chunks in parallel, verifies SHA-256 hashes locally, and assembles the file. For files already partially cached (delta sync scenario), the client only downloads chunks it doesn't already have.

Other Endpoints

POST /v1/files/share, Share a file or folder with specified users and permission level (view, edit, comment).

GET /v1/files/{file_id}/versions, List version history with timestamps, editor, and size delta.

POST /v1/files/move, Move or rename a file/folder. Updates parent_folder_id and triggers sync notification.

The data model separates into two distinct stores: PostgreSQL for structured metadata (files, versions, permissions, chunk references) and S3 for the actual chunk data. This split reflects their fundamentally different characteristics, metadata needs ACID transactions and complex queries while chunk data needs cheap, durable blob storage.

PostgreSQL Schema

1-- Users
2CREATE TABLE users (
3  user_id       UUID PRIMARY KEY,
4  email         VARCHAR(255) UNIQUE NOT NULL,
5  display_name  VARCHAR(100),
6  storage_quota BIGINT DEFAULT 2147483648,  -- 2GB default
7  storage_used  BIGINT DEFAULT 0,
8  plan_type     VARCHAR(20) DEFAULT 'free',
9  created_at    TIMESTAMPTZ NOT NULL
10);
11
12-- File tree (files and folders)
13CREATE TABLE files (
14  file_id          UUID PRIMARY KEY,
15  owner_id         UUID REFERENCES users(user_id),
16  parent_folder_id UUID REFERENCES files(file_id),
17  name             VARCHAR(255) NOT NULL,
18  is_folder        BOOLEAN DEFAULT FALSE,
19  current_version  INTEGER DEFAULT 1,
20  deleted_at       TIMESTAMPTZ,
21  UNIQUE(parent_folder_id, name)
22);
23
24-- Version history
25CREATE TABLE file_versions (
26  version_id   UUID PRIMARY KEY,
27  file_id      UUID REFERENCES files(file_id),
28  version_num  INTEGER NOT NULL,
29  size_bytes   BIGINT NOT NULL,
30  editor_id    UUID REFERENCES users(user_id),
31  created_at   TIMESTAMPTZ NOT NULL,
32  UNIQUE(file_id, version_num)
33);
34
35-- Content-addressable chunk registry
36CREATE TABLE chunks (
37  chunk_hash   VARCHAR(64) PRIMARY KEY,  -- SHA-256 hex
38  size_bytes   INTEGER NOT NULL,
39  ref_count    INTEGER DEFAULT 1,
40  created_at   TIMESTAMPTZ NOT NULL
41);
42
43-- Links versions to their chunks (ordered)
44CREATE TABLE version_chunks (
45  version_id    UUID REFERENCES file_versions(version_id),
46  chunk_index   INTEGER NOT NULL,
47  chunk_hash    VARCHAR(64) REFERENCES chunks(chunk_hash),
48  PRIMARY KEY(version_id, chunk_index)
49);
50
51-- Sharing permissions
52CREATE TABLE permissions (
53  permission_id UUID PRIMARY KEY,
54  file_id       UUID REFERENCES files(file_id),
55  user_id       UUID REFERENCES users(user_id),
56  access_level  VARCHAR(20) NOT NULL,  -- view, edit, comment
57  granted_at    TIMESTAMPTZ NOT NULL,
58  UNIQUE(file_id, user_id)
59);
60
61-- Upload session tracking (for resumable uploads)
62CREATE TABLE upload_sessions (
63  session_id    UUID PRIMARY KEY,
64  file_id       UUID REFERENCES files(file_id),
65  uploader_id   UUID REFERENCES users(user_id),
66  total_chunks  INTEGER NOT NULL,
67  status        VARCHAR(20) DEFAULT 'in_progress',
68  created_at    TIMESTAMPTZ NOT NULL,
69  expires_at    TIMESTAMPTZ NOT NULL  -- 24-hour TTL
70);
71
72-- Tracks which chunks are confirmed per session
73CREATE TABLE session_chunks (
74  session_id    UUID REFERENCES upload_sessions(session_id),
75  chunk_index   INTEGER NOT NULL,
76  chunk_hash    VARCHAR(64) NOT NULL,
77  etag          VARCHAR(64) NOT NULL,
78  uploaded_at   TIMESTAMPTZ NOT NULL,
79  PRIMARY KEY(session_id, chunk_index)
80);

S3 Storage

Chunks are stored in S3 with the SHA-256 hash as the object key: s3://dropbox-chunks/{chunk_hash}. This is content-addressable storage. The key is derived from the content itself. Two identical chunks uploaded by different users produce the same key and are stored exactly once.

Why This Schema Works

The ref_count on the chunks table is critical for garbage collection. When a file version is deleted (e.g., expired from version history), the system decrements the ref_count of each referenced chunk. When ref_count reaches zero, no file version references that chunk, and it can be safely deleted from S3. This prevents both premature deletion (a chunk still used by another file) and orphaned storage (chunks that no one references).

The files table uses a parent_folder_id self-reference to model the directory tree. The UNIQUE constraint on (parent_folder_id, name) prevents duplicate filenames within a folder, enforcing file system semantics at the database level.

Components

Client apps (desktop, mobile, web): The desktop client is the most sophisticated: it watches the local Dropbox folder for file changes, computes chunk hashes using SHA-256, maintains a local database of known chunk hashes for delta detection, and manages the upload/download queue. Mobile clients sync on-demand (pull individual files) rather than syncing the entire folder. The web client provides browser-based access with drag-and-drop upload.

API Gateway: Single entry point handling authentication (OAuth 2.0 tokens), rate limiting (token bucket per user), TLS termination, and request routing to backend services. The gateway also handles the long-poll connections for sync notifications, keeping connections open for up to 90 seconds.

File Service: Manages file metadata operations: create, rename, move, delete, share, and version listing. Reads and writes to PostgreSQL. Publishes file change events to Kafka for downstream consumers (Sync Service, CDN invalidation). Stateless. Any instance can handle any request.

Chunk Service: Handles the chunk-level operations: dedup checking (hash lookup against the chunks table), pre-signed URL generation for S3 uploads, upload session tracking, and commit verification. This service never handles chunk data directly. It only generates URLs that let clients talk to S3.

Sync Service: Maintains long-poll connections with clients and pushes change notifications when files are created, modified, or deleted. Consumes events from Kafka and fans them out to connected clients. Each client provides a cursor (sequence number) and the Sync Service returns all changes since that cursor. Uses consistent hashing to assign user-to-server affinity so a user's notification state lives on one Sync Service instance.

PostgreSQL: Stores all structured metadata: users, files, file versions, chunks (hash + ref_count), version_chunks, and permissions. Deployed with synchronous replication to a standby for zero data loss on primary failure. Read replicas handle the read-heavy metadata queries.

S3 (object storage): Stores chunk data with the SHA-256 hash as the object key. Content-addressable: the same bytes always map to the same key. Cross-region replication for durability. Lifecycle policies archive old versions to S3 Glacier after 90 days.

Kafka: Event bus connecting services. When a file is uploaded, the File Service publishes a change event. The Sync Service consumes it to notify clients. Other consumers handle audit logging, analytics, and CDN cache warming. Kafka decouples producers from consumers and absorbs traffic spikes during peak hours.

Redis: Distributed locking and lease management for Sync Service instances. Each instance holds a time-limited lease on its assigned user partitions, preventing split-brain scenarios during failover. Leases are stored with a TTL so that if an instance crashes, its lease expires automatically and another instance takes over.

CDN: Caches popular shared files at edge locations. When a file is shared with thousands of users (e.g., a company-wide document), the CDN serves subsequent downloads without hitting S3. Most personal files are not CDN-cached because they are accessed from a small number of devices.

Upload Flow (with Dedup)

1. User saves a 20MB file in their Dropbox folder. The desktop client detects the change via filesystem watcher.
2. Client splits the file into five 4MB chunks and computes SHA-256 hash of each chunk.
3. Client sends POST /v1/files/upload/init with the file name, parent folder, and list of five chunk hashes.
4. Chunk Service looks up each hash in the chunks table. Three of the five hashes already exist (from a previous version of the same file or from another user's identical content). The server returns only two missing chunk indices with pre-signed S3 URLs.
5. Client uploads the two missing chunks directly to S3 in parallel via the pre-signed URLs. Each upload is confirmed with an S3 ETag.
6. Client sends POST /v1/files/upload/commit with the upload session ID and chunk ETags.
7. Chunk Service confirms all chunks are present by verifying ETags against the upload session, then signals File Service to create the new file_version record. File Service creates the version record in PostgreSQL, linking to all five chunk hashes (three existing + two new). Increments ref_count for the three existing chunks. Sets current_version on the files record.
8. File Service publishes a "file_updated" event to Kafka with the file_id, version_num, and user_id.

Result: The user uploaded 8MB instead of 20MB. A 60% bandwidth reduction from dedup alone. If this was an edit to an existing file where only one chunk changed (delta sync), the savings would be even greater: 4MB instead of 20MB (80% reduction).

Delta Sync Flow

1. Device A has a 100MB file (25 chunks) synced. User edits the file, changing content in the middle.
2. Client rechunks the file and computes new hashes. 24 of 25 hashes are identical to the previous version. One chunk has a new hash.
3. Client sends upload/init with 25 hashes. Server identifies 24 as existing, returns one missing chunk.
4. Client uploads only the 1 changed chunk (4MB) and commits the new version.
5. Sync Service picks up the Kafka event and pushes notification to Device B via the long-poll connection.
6. Device B receives the notification, fetches the new file version metadata, and compares chunk hashes with its local copy.
7. Device B identifies 1 changed chunk, downloads just that 4MB chunk from S3, and reconstructs the updated file by combining 24 locally cached chunks with the 1 new chunk. Result: A 100MB file edit transferred only 4MB in each direction: 4MB upload from Device A and 4MB download to Device B. Without delta sync, it would have been 200MB of total transfer (100MB up + 100MB down).

Download Flow

1. User opens the Dropbox mobile app and taps on a file.
2. Client sends GET /v1/files/{file_id}/download.
3. File Service returns the current version's chunk hashes and pre-signed S3 download URLs.
4. Client checks its local chunk cache. Any chunks already downloaded (from previous versions or shared files) are skipped.
5. Client downloads missing chunks in parallel from S3 (or CDN for popular shared files).
6. Client verifies each chunk's SHA-256 hash matches the expected value, then assembles the file locally.

Chunking Strategy: Fixed-Size vs Content-Defined

The choice of chunking algorithm fundamentally affects delta sync efficiency.

Fixed-size chunking splits the file at every 4MB boundary. Simple to implement: chunk N starts at byte offset N * 4MB. The problem: inserting a single byte at the beginning of a file shifts every chunk boundary by one byte, causing every chunk to have different content and a different hash. A 1-byte insertion in a 100MB file invalidates all 25 chunks, forcing a full re-upload.

Content-defined chunking (CDC) uses a rolling hash (Rabin fingerprint) that scans the file byte by byte. When the hash of a sliding window matches a predefined pattern (e.g., lowest 13 bits are zero), that byte position becomes a chunk boundary. Chunk boundaries are determined by the local content, not by absolute position. An insertion shifts only the boundary of the chunk it falls within. All other boundaries stay put.

CDC produces variable-size chunks (typically 2-8MB with a 4MB average). The implementation is more complex: you need the Rabin fingerprint computation, min/max chunk size enforcement (to prevent tiny or huge chunks), and handling of the variable-size metadata. But the delta sync improvement is dramatic: an insertion anywhere in a 100MB file typically changes only 1-2 chunks instead of all 25.

Content-Addressable Deduplication Deep Dive

Every chunk is identified by its SHA-256 hash. This hash serves triple duty:

1. Dedup key: Before uploading, the client sends chunk hashes to the server. If a hash exists in the chunks table, that exact data is already in S3. No upload needed.
2. S3 object key: The chunk is stored at s3://dropbox-chunks/{sha256_hash}. Two users uploading identical files produce identical hashes and map to the same S3 object.
3. Integrity check: After downloading a chunk, the client recomputes the SHA-256 hash and verifies it matches. Any corruption (network, disk, or S3) is detected immediately.

SHA-256 collision risk: The probability of two different chunks producing the same SHA-256 hash is approximately 1 in 2^256 (roughly 10^77). For context, if you hashed every atom in the observable universe, the chance of a collision would still be negligible. No file storage system has ever observed a SHA-256 collision. This is not a theoretical concern in practice.

Conflict Resolution

When two devices edit the same file concurrently:

1. File is at version 5 on both Device A and Device B.
2. Both devices go offline and make different edits.
3. Device A reconnects first, uploads its changes. The server creates version 6 (version_num 5 + 1 succeeds because current version is 5).
4. Device B reconnects and attempts to upload based on version 5. The server detects that the current version is now 6, not 5, version mismatch.
5. The server rejects Device B's upload with a 409 Conflict response.
6. Device B's client saves its local changes as a conflicted copy: report (Device B's conflicted copy 2026-03-03).docx. The original file syncs to version 6 (Device A's changes).
7. The user sees both files and manually resolves by merging the changes or choosing one version.

Why not auto-merge? Dropbox handles arbitrary binary files (PDFs, images, executables), not just text. Auto-merging two versions of a JPEG or a compiled binary is meaningless. Even for text files, automatic merging requires understanding the file format (XML, JSON, plain text each have different merge semantics). The conflicted copy approach is format-agnostic and always safe. No data is ever lost.

Push vs Pull for Sync Notifications

Long polling (pull, chosen): The client sends a request that the server holds open for up to 90 seconds. If a change occurs, the server responds immediately with the update. If no change occurs, the server returns an empty response and the client reconnects. This works through all firewalls and proxies because it uses standard HTTP.

WebSockets (push): The server pushes change notifications to the client over a persistent bidirectional connection. Lower latency (no reconnection overhead) and more efficient for high-frequency updates. But WebSocket connections are blocked by many corporate proxies, require sticky sessions at the load balancer, and complicate horizontal scaling of the Sync Service.

Server-Sent Events (SSE): Unidirectional server-to-client stream over HTTP. Simpler than WebSockets, works through most proxies. But lacks bidirectional communication and has browser-specific limitations (6 connection limit per domain in HTTP/1.1).

Decision: Long polling for maximum compatibility. File sync events are infrequent (minutes apart, not seconds), so the 90-second polling cycle adds negligible overhead. The worst-case notification delay is 90 seconds, which is acceptable for file sync. Dropbox actually uses long polling in production for this reason.

Fixed-Size vs Content-Defined Chunking

Fixed-size (simpler): Every chunk is exactly 4MB (except the last). Trivial to implement: chunk index = byte offset / 4MB. Fast, predictable, easy to debug. Weakness: any insertion or deletion shifts all subsequent chunk boundaries, causing unnecessary chunk invalidation during delta sync.

Content-defined (better delta sync): Rabin fingerprint sets boundaries based on content. Insertions only affect the local chunk. But implementation is more complex, chunk sizes vary (need min/max enforcement), and the rolling hash computation adds CPU overhead on the client.

Decision: Start with fixed-size chunking for simplicity. Most file edits (appending to a document, modifying a section) change content at the end or in a localized region, fixed-size chunking handles these well. Content-defined chunking is a Phase 2 optimization for power users with large files and frequent mid-file insertions. Dropbox started with fixed-size and later moved to content-defined chunking.

PostgreSQL vs DynamoDB for Metadata

PostgreSQL (chosen): ACID transactions for atomic version commits. Foreign keys enforce referential integrity (no orphaned file records). Complex queries for directory listings, permission checks, and version history. The 50TB total metadata is manageable with partitioning and read replicas.

DynamoDB: Better horizontal scaling and single-digit millisecond latency on key-value lookups. But: no transactions across items (version commit + chunk ref_count updates would be non-atomic), no foreign keys, no complex joins for directory tree queries. You would need to denormalize everything and accept eventual consistency.

Decision: PostgreSQL. The metadata workload is relational (files belong to folders, versions reference chunks, permissions link users to files). The strong consistency requirement for file operations eliminates eventually-consistent stores. The total data size (50TB) is within PostgreSQL's range with proper partitioning.

Chunk Corruption

A chunk stored in S3 becomes corrupted due to bit rot, disk failure, or a rare S3 storage error. A client downloads the chunk and the SHA-256 hash does not match the expected value.

Detection: The client computes SHA-256 of every downloaded chunk and compares it to the expected hash from the file version metadata. A mismatch means corruption. This check is built into the download flow, corruption is detected immediately, never silently.

Recovery: The client reports the corrupted chunk hash to the server. The server fetches the chunk from S3's cross-region replica (S3 stores data across at least 3 availability zones). If the replica is also corrupted (extremely unlikely with 11 nines durability), the server checks if any other S3 region has a valid copy. The client retries the download with the replacement chunk.

Prevention: S3 performs internal integrity checks using checksums on every read and write. S3's own repair mechanisms detect and replace corrupted objects automatically. The client-side SHA-256 verification is a defense-in-depth layer.

Metadata Database Failure

The PostgreSQL primary goes down.

Impact: All write operations (uploads, commits, file changes) fail immediately. Read operations continue on read replicas until replication lag causes staleness.

Recovery: The synchronous standby is promoted to primary automatically via Patroni or similar HA manager. Zero data loss for committed transactions because the standby has all committed WAL entries. Failover time: 10-30 seconds. During failover, clients experience upload failures but retry automatically. In-flight upload sessions are preserved, chunks already in S3 are not affected, and the commit can be retried against the new primary.

Prevention: Synchronous replication to at least one standby. Cross-region async replication for disaster recovery. Regular backup verification by restoring to a test cluster and running integrity checks.

Sync Service Downtime

All Sync Service instances go down. Clients cannot receive change notifications.

Impact: File changes are still being recorded (uploads succeed, PostgreSQL has the data, Kafka has the events). But clients do not learn about changes until the Sync Service recovers. Files appear stale on devices that haven't made changes.

Graceful degradation: Desktop clients fall back to periodic polling. Every 5 minutes, they query the File Service directly for changes since their last known cursor. This is less efficient (more API calls, higher latency) but ensures sync continues even without the long-poll infrastructure. The client detects Sync Service unavailability when long-poll connections consistently fail and automatically switches to poll mode.

Recovery: When the Sync Service recovers, it replays unprocessed events from Kafka (Kafka retains events for 7 days). Clients reconnect with their last cursor and receive all missed changes in a batch. No data is lost. The events were durably stored in Kafka throughout the outage.

Resumable Upload Recovery

A user uploading a 2GB file (500 chunks) loses connectivity after 300 chunks are uploaded.

State preservation: Each successfully uploaded chunk was confirmed by S3 and recorded in the upload session table (session_id, chunk_index, etag, uploaded_at). The session has a 24-hour TTL.

Recovery flow: When the client reconnects, it sends GET /v1/uploads/{session_id}/status. The server returns the list of confirmed chunk indices (0-299). The client resumes uploading from chunk 300. Only 200 remaining chunks need to be uploaded, saving 60% of the bandwidth.

Edge case: If the client reconnects after the 24-hour TTL, the session has expired. The client initiates a new upload session with the same chunk hashes. The dedup check identifies all 300 previously uploaded chunks as already existing in S3, so only the remaining 200 are uploaded. The result is identical. The chunks are already in S3 regardless of whether the session is active.

Split Brain in Sync Service

Two instances of the Sync Service both believe they own the same user's notification state (due to a network partition during consistent hashing ring rebalancing).

Impact: The user receives duplicate notifications (both instances push the same change) or missed notifications (each instance thinks the other is handling it).

Mitigation: The Sync Service uses a lease-based ownership model: each instance holds a time-limited lease on its assigned users (stored in Redis with a TTL). If an instance crashes or is partitioned, its lease expires and another instance takes over. During the brief overlap, duplicate notifications are harmless (the client is idempotent, receiving the same change notification twice just triggers a no-op sync check). The client's cursor-based sync protocol ensures no changes are missed regardless of notification delivery.

Real-Time Collaborative Editing

Extend Dropbox beyond file sync to support Google Docs-style real-time collaboration on text documents and spreadsheets. This requires a fundamentally different architecture from file-level sync.

Key additions:

1. Conflict resolution at the character level using Operational Transformation (OT) or CRDTs instead of file-level version counters. Each keystroke is an operation (insert character at position N) that can be transformed against concurrent operations from other users.
2. Sub-100ms latency for operation delivery. Long polling is too slow. This requires WebSocket connections to a collaboration server that broadcasts operations to all participants.
3. Cursor presence: each collaborator's cursor position and selection are shared in real-time, showing who is editing where.
4. Bridge to file sync: when a collaborative editing session ends, the final state is saved as a new file version that syncs to non-collaborating devices through the existing chunk-based sync pipeline. This is essentially building a separate product (like Dropbox Paper) that integrates with the existing file storage infrastructure.

Selective and Smart Sync

Not every device needs every file. Smart sync extends the current on-demand mobile behavior to desktop.

Virtual files: The desktop client shows all files in the Dropbox folder but only downloads content when a file is opened. Files that haven't been accessed recently exist as lightweight placeholders (using platform-specific APIs like Windows Cloud Files or macOS File Provider). This dramatically reduces local disk usage.

Predictive pre-fetch: Machine learning model predicts which files a user is likely to open next based on access patterns (e.g., always opens "weekly-report.xlsx" on Monday mornings). Pre-fetches those files in the background so they open instantly when needed.

Policy-based sync: Enterprise admins set rules like "always sync files in the Finance folder" or "never sync video files over 1GB on mobile." Policies override the default behavior for specific user groups or device types.

End-to-End Encryption

Current architecture: Dropbox encrypts files at rest (S3 server-side encryption) and in transit (TLS). But Dropbox holds the encryption keys and can theoretically access file contents.

Zero-knowledge encryption: The client encrypts chunks before uploading using a key derived from the user's password. Dropbox servers only see encrypted blobs and cannot decrypt them. This is significantly more complex:

1. Key management, losing the password means losing all data (no server-side password reset). Need a recovery key mechanism.
2. Sharing, encrypted files shared with another user require key exchange. The recipient must receive a decryption key through a secure channel.
3. Dedup impact, encrypted chunks from different users produce different hashes even for identical content, breaking cross-user dedup. Trade-off: security vs storage efficiency. Convergent encryption (encrypting with a hash of the plaintext) preserves dedup but leaks information about duplicate content.

Full-Text Search

Index the contents of text-based files (documents, PDFs, code, spreadsheets) to enable searching across a user's entire Dropbox.

Architecture: An indexing pipeline processes uploaded files: extract text content (Apache Tika for format detection), tokenize and analyze (language detection, stemming), and write to an Elasticsearch index partitioned by user_id. Search queries are scoped to the user's files and shared files they have access to. Privacy-sensitive: the search index must respect sharing permissions and be purged when files are deleted or access is revoked.