Requirements

Functional Requirements:

• Enable real-time collaborative editing where multiple users can edit the same document and see each other's changes instantly.
• Support offline editing with automatic synchronization of changes when the connection is restored.
• Enable commenting on text ranges with threaded replies for asynchronous review workflows.
• Enable sharing and permission controls, allowing owners to add viewers or editors and create shareable links.
• Store a revision history of every change, allowing users to view or restore past versions.

Non-Functional Requirements:

• Low latency: Keystroke-to-screen for remote collaborators < 100ms (same region), <300ms (cross region)
• Consistency: All users must converge to the same document state (strong eventual consistency)
• Availability: 99.99% uptime -- users expect docs to always be accessible
• Durability: Zero data loss - every acknowledged edit is persisted
• Scalability: Support millions of documents, thousands of concurrent editors per popular document
• Offline-first: Edits made offline must merge cleanly without data loss.
• Security: End to end encryption optional, RBAC enforced at every API call, audit logging

API Design

Document CRUD

POST /v1/documents

Body: {"title": "Untitled, "owner_id": "user123"}

Response: {"doc_id": "doc_abc", "created_at": timestamp}

GET /v1/documents/{doc_id}

Response: {"doc_id", "title", "content": , "version": int, "permissions": {...}}

Real-time editing (WebSocket)

WS /v1/documents/{doc_id}/collaborate

• Client sents: {"type": "operation", "ops": [...], "base_version": int, "client_id": "c1"}
• Server sends: {"type": "operation", "ops": [...], "version" int, "author_id": "user456"}
• Server sends: {"type": "ack", "version": int}
• Server sends: {"type": "presence", "user_id": "user456", "cursor": {"index":42}}

Comments

POST /v1/documents/{doc_id}/comments

Body: {"anchor": {"start": 10, "end": 25}, "body": "Rephrase this?", "author_id": "user456"}

POST /v1/documents/{doc_id}/comments/{comment_id}/replies

Body: {"body": "Done!", "author_id": "user123"}

Sharing & Permissions

PUT /v1/documents/{doc_id}/permissions

Body: {"grants":[{"user_id": "user789", "role": "editor"}]}

POST /v1/documents/{doc_id}/share-link

Body: {"role": "viewer", "expires_at": timestamp}

Response: {"link": "https://docs.example.com/s/abc123"}

Revision History

GET /v1/documents/{doc_id}/revisions?limit=50&cursor={cursor}

Response: {"revisions": [{"version": 42, "author_id", "timestamp", "summary"}]}

POST /v1/documents/{doc_id}/restore

Body: {"version": 42}

High-Level Design

The core challenge is conflict resolution - multiple users editing the same position simultaneously. The industry standard solutions are:

• Operational transformation (OT) - Google Docs' original approach. A central server transforms concurrent operations against each other.
• CRDTs (Conflict free replicated data types) - Newer approach (used by Figma, Yjs). Operations commute without a central coordinator.

We choose OT with a central coordination server per document because:

1. Its proven at Google-scale for text editing
2. Simpler server side logic than distributed CRDTs
3. Central server enables easy version numbering for history.

Components:

1. Client editor - rich text editor with local OT engine. Applies local ops immediately, buffers and sends to server.
2. Collaboration Service - WebSocket server that coordinates OT per document. One logical "owner" per document (can be sharded).
3. Document Service - CRUD, permissions, metadata. Stateless REST API.
4. Persistence Layer - Stores document snapshots -> operation log
5. Presence Service - Broadcasts cursor positions and online status via pub/sub
6. Comment Service - Manages anchored comments with position tracking as document changes.
7. Revision Service - Periodically snapshots document state, provides diff/restore
8. Permission Service - RBAC, share links, enforcement at gateway level.
9. Offline Sync Queue - Client-side operation buffer that replays on reconnection

Detailed Component Design

1. OT Collaboration Engine

The core algorithm:

Each document has a linear version number. Every operation (insert, delete, format) is submitted against a base_version.

Client A (version 5): insert("X", pos=3) -> sends to server

Client B (version 5): insert("Y", pos=1) -> sends to server

Server receives A's op first, applies it -> version 6. When B's op arrives (base_vresion=5, but server is now at 6), the server transforms B's op against A's op:

• A inserted at pos 3, B inserted at pos 1. Since B's position < A's position, B's op is unchanged.
• If B had inserted at pos 4, it would be transformed to post 5 (shifted by A's insert).

Server state machine per document:

1. Receive op from client with base_version
2. If base_version == current_version: apply directly
3. If base_version < current_version: transform op against all ops between base_version and current_version
4. Apply transformed op, increment version
5. Broadcast transformed op to all other connected clients
6. Send ACK with new version to the submitting client

Client state machine:

• Synchronized: No pending ops. Apply remote ops directly
• Awaiting ACK: local op sent, not yet acknowledged. Buffer incoming remote ops, transform against pending op.
• Awaiting ACK -> Buffer: New local ops generated while waiting. Queue them. transform against both pending and remote.

Scaling: Each document is assigned to one collaboration server instance (consistent hashing on doc_id). If a document gets too hot (1000+ concurrent editors), the server can shard by section/paragraph - but this is rare.

Persistence: Every operation is appended to an operation log (append only, Kafka or DDB stream). Periodically (every N ops or T seconds), a snapshot is written. Recovery = load latest snapshot + replay ops since.

2. Offline Sync & Conflict Resolultion

Problem: User edits offline for hours, then reconnects. Their operations are based on a stale version.

Design:

• Client stores all local operations in IndexedDB with their base_version.
• On reconnect:

1. Client sends its last_known_version to the erver.
2. Server responds with all operations from last_known_version to current_version
3. Client transforms its buffered local ops against the server ops (client side OT).
4. Client sends transformed ops to server as a batch
5. Server applies them (may need further transformation if new ops arrived during sync)

Guarantee: OT's transformation properties ensure convergence regardless of how long the client was offline. No manual conflict resolution needed for text - the math handles it.

Edge case - large divergence: If the client has been offline for days and the op log is huge, the server sends a full snapshot instead of the op stream. Client rebases local ops against the snapshot diff.

3. Revision history and snapshots

Problem: Storing every keystroke forever is expensive. Users want meaningful "versions" they can browse and restore.

Design - Two-level storage:

Op log - every individual operation - 30 day retention - PurposeL real-time sync, recent undo

Snapshots - Full document state - forever - revision history, restore points

Snapshot strategy:

• Auto snapshot: every 100 ops or 5 minutes of inactivity (whichever comes first)
• Named snapshots: User explicitly "saves a version" or the system detect a "session boundary" (all editors leave, then someone returns)
• Compaction: After 30 days, the op log between snapshots is discarded, only snapshots remain.

Viewing history: The revision timeline shows snapshots with metadata (author, timestamp, change summary). Summary is generated by diffing consecutive snapshots.

Restoring: Creates a new operation that sets the document content to the snapshot state. This is itself an operation in the log - so "restore" is undoable and doesn't destroy subsequent history.

Storage:

• Op log: DynamoDB (partition key = doc_id, sort key = version). High write throughput, TTL for auto expiry.
• Snapshots: S3 (immutable blobs, keyed by doc_id/version). Metadata in DDB.

Session cache (Redis)

Purpose: Hold ephemeral, hot state that doesn't belong in durable storage

What lives here:

• doc:{doc_id}:version - current version number - TTL: While doc has active connections
• doc:{doc_id}:ops:{v} - recent ops (last ~100) - TTL: 1 hour
• doc:{doc_id}:sessions - Set of connected {user_id, client_id, cursor_pos} - Heartbeat refreshed, 30s TTL
• doc:{doc_id}:lock - which collaboration server owns this doc - lease based, 30s TTL + renewal

On a cache miss: Fall thru to dynamodb for the op log an latest snapshot. Rebuild in-memory state.

Scaling: Redis cluster sharded by doc_id. Each doc's keys land on the same shard.

Metadata Store (PostgreSQL)

Purpose: Durable source of truth for everything about documents that isn't the document content itself.

Schema:

documents (

doc id

title

ownerid

created_at

updated_at

current_version

storage_region

)

permissions (

doc_id

grantee

role

granted_by

granted_at

PRIMARY_KEY (doc_id, grantee)

)

share_links

comments

Comment anchor tracking: When the document changes, comment anchors drift. Two approaches:

1. Lazy rebase: store anchor_version. On read, transform the anchor position thru all ops from anchor_version to current_version. Cache the rsult.
2. Eager update: CS updates anchor positions in Postgres as ops are aplied. simpler reads but more write load.

We use lazy rebase

Why Single Owner is Required for OT

OT relies on a total ordering of operations. The server assigns sequential version numbers. Without the lock, the document corrupts

What the lock actually enforces

Server A holds lock for doc_123 (lease: 30s, renewed every 10s)

Server B receives a WS connection for doc_123

Server B checks Redis -> lock held by A -> routes client to A (or rejects + tells client to reconnect to A)

If Server A dies:

1. Lease expires after 30s (no renewal)
2. Server B acquires lock via SETNX
3. Server B loadds state from Redis cache + DDB
4. Server B becomes the new owner, accepts connections

Sticky Routing Layer

The WebSocket Gateway needs to route clients to the correct collaboration server. This is done via:

1. Consistent hashing on doc_id --> gives a preferred server (fast path, no lock check needed)
2. Lock as fallback --> if the preferred server is down or the ring rebalanced, the lock in Redis is the source for who actualy owns the doc

So the routing is:

Client connects for doc_123

• Gateway hashes doc_123 -> Server A (preferred)
• Gateway checks: is A healthy? Yes -> Route to A
• If A is dead: wait for lock expiry -> new server acquires -> route there

Could you avoid the single owner constraint?

Yes - by switching from OT to CRDTs. Google chose OT because the single-owner constraint is acceptable -failover in 30s is fine for a document editor, and the simpler version model makes history and snapshots trivial. If you needed sub-second failover (e.g. multiplayer gaming) CRDTs would be the better choice.